A few days ago I wrote about my problems with the Directory Assistance and getting user roles with context.getUser().

Prompted by Mark Leusink's comment, I decided to take a look at our Directory Assistance settings. I started changing settings and... context.getUser().getRoles() started to work!

The setting that made all the difference was "Use exclusively for group authorization or credential authentication".

If you set this to Yes, as it was originally set in our case, context.getUser().getRoles() will not work. Setting this to No makes it work. No other settings have any influence on context.getUser().

So, whether you have access to Directory Assistance settings and / or are allowed to change them will decide what implementation you choose. context.getUser().getRoles() is shorter and more elegant, session.getCurrentDatabase().getACL().getEntry(session.getEffectiveUserName()).getRoles() seems to be safer in the long run. The choice is yours.

Permalink | Leave a comment  »

As many of you have experienced, date and number formatting of the link text in xp:link does not work. Sure, you can assign a data converter, just as you would for a field or a computed text, but it won't work. Paul Withers and Frédéric Dehédin, among others, have offered nice SSJS solutions to this problem.

When I encountered the same problem, I took a somewhat different approach. I didn't like the idea of creating SSJS libraries for something that is essentially a hack and which can be easily solved with a single line (albeit long) of Java code.

So here is your one-liner that will let you format dates in xp:links:

java.text.DateFormat.getDateInstance().format(yourDateValue);

What happens here is that you use the DateFormat class (you must call it by its full name java.text.DateFormat) which knows how to format dates. Using DateFormat, you first get dateInstance which knows everything about dates and their formatting and then you call format with the date value that you want to format.

Let's first take a look at the date value. This value needs to be of type java.util.Date.

If you already have this value in a field on your xpage, you simply need to call

getComponent("elementID").getValue()

The complete code would look something like this:

java.text.DateFormat.getDateInstance().format(getComponent("elementID").getValue());

You can also read a date value from the Notes back-end, a Notes document for example. Say you have a Notes document as the data source and you call it doc. You'll get your date value by calling:

doc.getItemValueDate("fieldName")

which translates to:

java.text.DateFormat.getDateInstance().format(doc.getItemValueDate("fieldName"));

The method getItemValueDate returns java.util.Date, so you are OK there. If you, however, have a Notes DateTime item, you need to convert it to Java Date first. You do so by calling the method toJavaDate(). For example:

doc.getItemValueDateTime("fieldName").toJavaDate()

getItemValueDateTime returns Notes DateTime and by calling toJavaDate you transform it to Java Date that you can use.

That was the date value. Now, let's look at the format options. Calling:

java.text.DateFormat.getDateInstance().format(doc.getItemValueDate("fieldName"));

will return you a string formatted using the default date format and the default locale for the JVM running your application. Most likely, this is not what you want. Fortunately, you can change this easily. getDateInstance accepts arguments that tell it what date format and what locale you want to use in the form of getDateInstance(int style, Locale aLocale). For example, you could do something like this:

java.text.DateFormat.getDateInstance(java.text.DateFormat.SHORT, Locale.US).format(doc.getItemValueDate("fieldName"));

or

java.text.DateFormat.getDateInstance(java.text.DateFormat.LONG, Locale.GERMANY).format(doc.getItemValueDate("fieldName"));

As you can see, you can use predefined constants for format and locale. The format constants are: FULL, LONG, MEDIUM and SHORT. Just don't forget you need to call them by the full name, e.g. java.text.DateFormat.FULL.

There are many predefined locales as well, you should check Java API documentation (look under Field Summary) to find out more. If your locale is not predefined, don't despair - it is easy to define it. What you would need to know are the lowercase two-letter ISO-639 code for the language and the uppercase two-letter ISO-3166 code for the country. So, if you want to specify locale for the Italian speaking part of Switzerland, you would use:

new Locale("it", "CH")

which in turn would give something like this:

java.text.DateFormat.getDateInstance(java.text.DateFormat.LONG, new Locale("it", "CH")).format(doc.getItemValueDate("fieldName"));

Of course, you could read language and country from some sort of settings document, making your display fully customizable.

Finally, you can also read the locale from the user's browser by calling:

context.getLocale()

resulting in:

java.text.DateFormat.getDateInstance(java.text.DateFormat.MEDIUM, context.getLocale()).format(doc.getItemValueDate("fieldName"));

And that's all. Let me know if something is unclear. And for your homework you do similar conversion for numbers using either the NumberFormat class or the String.format method.

Permalink | Leave a comment  »

It often happens that some parts of the GUI should be hidden, based on the current users authorizations. For that purpose, we have been using ACL roles and a short line of code to evaluate whether the current user is assigned to a specific role:

context.getUser().getRoles().contains("[RoleName]");

And this had worked very well until we recently introduced Directory Assistance and allowed users from the secondary directory to log in and use the application. For users from the secondary directory, the above procedure did not work - they could never see parts of the GUI requiring a role membership, i.e. the above line of code never evaluated to true. At the same time, users from the primary directory had no problem at all.

I have recently read Tim Tripcony's post on problems with page ACL and Directory Assistance and suspected a similar problem here.

This is what a simple test reveals. If a user from the primary directory logs in, context.getUser() will return a wealth of information:

context.getUser().getRoles() returns all roles assigned to the user and hence checking for a specific role works as expected.

But, this is what happens when a user from a secondary  directory logs in:

The context.getUser() is all by empty, no roles are returned and check for a role name will always return false.

And that is not good. I don't know if this is intended or not (feature or bug), but I needed to fix it. The first idea that occurred to me was to define roles for each user in a settings document, but that was too much of a hack for my liking. So I tested different ways to get the information about a user's roles and found that this one works:

session.getCurrentDatabase().getACL().getEntry(session.getEffectiveUserName()).getRoles()

This is how it looks if a user from the secondary directory logs in now:

So, if you are using or planning to use Directory Assistance and need to check assigned roles, this is the code that you can use.

Permalink | Leave a comment  »

The extension library provides the Dojo List Text Box (djextListTextBox) control that, connected to a value picker, makes it possible to create a list whose members are easy to remove. Using the oneUI v2.1, the default display is similar to this:

With some CSS, it is actually quite easy to accomplish. First, put your djextListTextBox inside a panel and give it a style class, let's call it verticalStack. Then, edit your existing or create a new style sheet and add following:

 .verticalStack span {
display: block;
margin-left: 0px !important;
}
.verticalStack a span {
display: inline;;
}

What does it all mean? If you take a look at the HTML code that the Dojo List Text Box control generates, you'll see that each list member is actually a <span> tag, which in turn contains an <a> tag that contains a <span> tag.

So, the CSS says that each <span> in the <div>  with class verticalStack should have block display style. <span> tags usually have inline display style, which means that multiple <span> tags are displayed in a row, one after the other. display: block will force them to behave like <p> or <div> tags and display one beneath the other.

Now, there is a <span> within the <a> tag. And it is also affected by the block display. And that makes the close button (little x) to be displayed beneath the rest of the list member. This is taken care of by the .verticalStack a span style, which returns normal, inline, display style to all <span> tags that are within a <a> tag that is within an element with .verticalStack class applied.

Finally, using the original style, each list member has applied left margin of 5 pixels to make separetion between consecutive members.We use margin-left: 0px !important to remove that margin and make our members nicely align. The CSS is hierarchical and there are styles higher in the hierarchy that would take precedence, so we need to use !important.

That's all that is needed. Have in mind that this works if you are using oneUI, but it should point you in the right direction even if you are using other layout. And, by applying some more CSS you can easily make it look even better:

Permalink | Leave a comment  »

I need expert help and advice.

In the application we are building, among other things, we keep record of all payments. One of the resources is an XPage that shows all payments due within certain number of days from 'today'.

This is how we did it: we already have a Domino view that displays all unpaid payments. We don't like creating a Domino view for each new requirement, so we decided to reuse this one. I wrote some Java code to go through all of these payments, find those that fall within the given time period and then extract interesting data (i.e. client name, due amount, due date) into a hash map. This hash map is saved in the view scope and used as data source for a repeat control on the XPage.

We also evaluated a solution with full-text database search, but one of the requirements is that when a new payment is added or a payment is paid, the XPage must display those changes immediately. Which means that delay between payment change and full-text index update is unacceptable. And we didn't think that constantly forcing index update was a good idea either.

Our current solution works well, but I don't like to go through all documents each time I reload the XPage. Now, the view is small and lean (only one sorted column) and it will hardly ever have more than a hundred documents, so the computation is quick. Still, I would like to perform the computation only when the view has changed. One of the solutions that occured to me is this:

1. Get view.
2. Get all column values for the first column (view.getColumnValues(0);).
3. Calculate the hash value of the column values.
4. Get previously calculated hash value from the session/application scope and compare the two.
5. If the hash values are the same, do nothing - the existing hash map with payment details is still valid.
6. If the hash values are different, perform full computation, update the hash map with payment details and update the hash value.

Is this a sound approach? It should be less intensive than the current solution, but is there a better way to do this? How expensive is the view.getColumnValues() call?

Any comments would  be greatly appreciated.

Permalink | Leave a comment  »

Disclaimer: The credit for this guide goes to the guys who figured out all the difficult parts: Jake Murzy and Bill Carroll for solving Red Hat related issues and Declan Sciolla-Lynch for showing how to integrate Apache and Domino LDAP. I only brought it all together in one blog post.

If you are using Mercurial as a source control system, you probably wish you had a central repository with which all of the developers could sync. You might use one of the public domain servers, but chances are that you would want your own server that is completely under your control. Declan has showed how to install and configure Mercurial on Ubuntu server. But, what if you use Red Hat or some of its clones, like CentOS or Scientific Linux? Mercurial, being built for Debian, won't install on a Red Hat-based system and. But, don't despair - it is actually quite easy to install it on Red Hat. Just follow the steps.

This guide is tested using Red Hat, CentOS and Scientific Linux, both versions 5.x and 6.x, but only 32-bit. The rest is based on CentOS 6, but you can choose whicever you prefer, as there is no difference between them for our purpose.

I won't go into installation of the server itself. It has become straight forward, even for Linux beginners. When you install, it is enough to choose Basic Server as a the installation type. This will install basic things, without much overhead. If you wish, choose Gnome / KDE desktop and X Window System (and anything else for that matter), so that you can have GUI. You won't need GUI to install Mercurial, though.

Even though it is not advisable, I have logged in as root and performed all of the following tasks.

I suggest that you update complete system before proceeding, simply issue the following in terminal:
# yum update

OK, now it's time to do some business. First let's install Apache server and confirm version

# yum install httpd

# httpd -v
Server version: Apache/2.2.15 (Unix)
Server built:   Jul  7 2011 11:27:40

Now, the Python should already be installed. Let's check the version:
# python -V
Python 2.6.5

You can have several Python environments installed on the server (something like JRE), depending on what different applications demand. Mercurial works fine with this version and we won't be running any conflicting applications, so we'll use this system installation.

We'll be building Mercurial and mod_wsgi from source, so we need some tools and libraries.
Make should already be installed on the system, let's check:
# make -v
GNU Make 3.81

If not, install by running
# yum install make

Now, install gcc
# yum install gcc

Install rpm-build
# yum install rpm-build

Install the libraries:
# yum install httpd-devel
# yum install python-devel

Now cd to a temporary / download directory. I usually use /tmp for this:
# cd /tmp

Download mod_wsgi source:
# wget http://modwsgi.googlecode.com/files/mod_wsgi-3.3.tar.gz

Unpack:
# tar xvfz mod_wsgi-3.3.tar.gz

Build the module:
# cd mod_wsgi-3.3
# ./configure
# make
(You will see some warnings here, but it is OK.)
# make install

This will install the library mod_wsgi.so in /etc/httpd/modules.

Now we need to do the similar thing with docutils.

# cd /tmp
# wget http://prdownloads.sourceforge.net/docutils/docutils-0.8.1.tar.gz?download
# tar xvfz docutils-0.8.1.tar.gz
# cd docutils-0.8.1
# ./setup.py install

This will install docutils in /usr/lib/python2.6/site-packages/docutils.

We can now finally start installing Mercurial. We are going to make an rpm file, which you can keep and use to install Mercurial on other servers.

# cd /tmp
# wget http://mercurial.selenic.com/release/mercurial-2.0.2.tar.gz 
# tar xvfz mercurial-2.0.2.tar.gz

Now you need to edit the file mercurial-2.0.2/contrib/mercurial.spec. Open this file in text editor and search for the line starting with "Version:" (near the top of the file) and replace "snapshot" with "2.0.2".
Then, go several lines down until you come to line starting with "BuildRequires:". Remove "python-docutils >= 0.5" from the list, so that the line looks like this:
BuildRequires: python >= 2.4, python-devel, make, gcc, gettext
(For some reason, rpm-build does not see the docutils that we built in previous step, so we need to remove it as a prerequisite. But, it is there and Mercurial will use it.)

If you are feeling geeky, you can do the same by issuing the following in the terminal:
# cd /tmp/mercurial-2.0.2/contrib/
# sed -i -e's/snapshot/2.0.2/' mercurial.spec
# sed -i -e's/, python-docutils >= 0.5//' mercurial.spec

Piece of cake :-)

Now, we need to repackage everything:
# tar cvf - mercurial-2.0.2 | gzip > mercurial-2.0.2.tar.gz

Create the rpm:
# rpmbuild -tb mercurial-2.0.2.tar.gz

If everything goes well, you'll see
+ exit 0
as the last line. We have our rpm now, we are almost there.

The created rpm could be in a few places. If you were doing everything like me and as the root, then yours should be in /root/rpmbuild/RPMS/i386/:
# ls /root/rpmbuild/RPMS/i386/
mercurial-2.0.2-0.i386.rpm

Now we can install and check version:
# rpm -ivh /root/rpmbuild/RPMS/i386/mercurial-2.0.2-0.i386.rpm
Preparing...                ########################################### [100%]
   1:mercurial              ########################################### [100%]
[root@code1 tmp]# hg --version
Mercurial Distributed SCM (version 2.0.2)

And that was installation done. Now it's time to configure it.

Open /etc/httpd/conf/httpd.conf in a text editor (you might want to make a copy first).

First, set these values (I show you my values, you should enter yours):
ServerAdmin admin@cs-computing.com
ServerName code1.cs-computing.com:80
NameVirtualHost *:80

(You will need to uncomment ServerName and NameVirtualHost)

Find section with number of LoadModule lines. Add the following:
LoadModule wsgi_module modules/mod_wsgi.so

Find section with AddHandler lines. Add the following:
AddHandler wsgi-script .wsgi

(In case that you installed specific version of Python to be used with Mercurial, add something like this as well:
WSGIPythonHome /usr/lib/python2.7
)

Now, scroll all the way to the end, and let's add following VirtualHosts:

First, an empty declaration, to trap requests with unrecognized host name:
<VirtualHost *:80>
</VirtualHost>

Then, add a declaration for our Mercurial server (in my environment, the server's host name is code1.cs-computing.com. However, I also created a DNS Alias hg.cs-computing.com that points to code1.cs-computing.com and which I use to access the Mercurial server):
<VirtualHost *:80>
    ServerName hg.cs-computing.com
    ErrorLog /var/log/httpd/hg.cs-computing.com-error_log
    CustomLog /var/log/httpd/hg.cs-computing.com-access_log common
    WSGIScriptAlias / /var/www/vhosts/hg.cs-computing.com/cgi-bin/hgweb.wsgi
</VirtualHost>

We need to create directories to hold our Mercurial repositories (repos) and configurations (cgi-bin):
# cd /var/www/
# mkdir -p vhosts/hg.cs-computing.com/cgi-bin
# mkdir -p vhosts/hg.cs-computing.com/repos

Copy file hgweb.wsgi from unpacked Mercurial source package to the cgi-bin directory:
# cp /tmp/mercurial-2.0.2/contrib/hgweb.wsgi /var/www/vhosts/hg.cs-computing.com/cgi-bin/

Open hgweb.wsgi in text editor and edit line starting with config to read:
config = "/var/www/vhosts/hg.cs-computing.com/cgi-bin/hgweb.config"

Create new file called hgweb.config in /var/www/vhosts/hg.cs-computing.com/cgi-bin:
# touch vhosts/hg.cs-computing.com/cgi-bin/hgweb.config

Open this file in text editor and add the following:
[paths]
/ = /var/www/vhosts/hg.cs-computing.com/repos/**
[web]
allow_push = *
push_ssl = false

We are nearing the end, hold on! :-)

It' time to create a test repo.
# cd /var/www/vhosts/hg.cs-computing.com/repos/
# hg init hgtest

We need to change owner and permissions (since we are running this as root):
# chgrp -R apache hgtest/
# chmod -R g+rwx hgtest/

Start the Apache:
# service httpd start
Starting httpd:                                            [  OK  ]

Open your browser and enter the address that you used for your Mercurial virtual host (hg.cs-computing.com in my case). You should see the Mercurial repos:
 

So, now you have working Mercurial server. We only need to add some sort of authentication. Let's use Domino LDAP, just like Declan did in his series. There is no special steps required to configure Domino LDAP, it should just work.

Open /etc/httpd/conf/httpd.conf in a text editor and scroll down to the definition of Mercurial virtual host. Add directory specification to the VirtualHost specification, so that it looks like this one:
<VirtualHost *:80>
    ServerName hg.cs-computing.com
    ErrorLog /var/log/httpd/hg.cs-computing.com-error_log
    CustomLog /var/log/httpd/hg.cs-computing.com-access_log common
    WSGIScriptAlias / /var/www/vhosts/hg.cs-computing.com/cgi-bin/hgweb.wsgi
    <Directory "/var/www/vhosts/hg.cs-computing.com/">
        AuthType Basic
        AuthBasicProvider ldap
        AuthzLDAPAuthoritative on
        AuthName "Mercurial Server: Login with your Domino credentials"
        AuthLDAPURL "ldap://ldap.cs-computing.com:389/O=Computing?CN"
        Require valid-user
    </Directory>
</VirtualHost>

Some explanations:
AuthType Basic says how to send password. Basic means in clear text, so this is not the most secure environment, but enough for our testing.
AuthBasicProvider ldap says Apache to use LDAP for authorization.
 AuthzLDAPAuthoritative on means that LDAP is only way to authorize with this server. If we had some other mean of authentication (e.g. Apache built-in users), that we could set this to off and the Apache would first try with LDAP and if it fails continue with other options. (BTW, this is default).
AuthName "Mercurial Server: Login with your Domino credentials" - this will be shown on login window.
AuthLDAPURL "ldap://ldap.cs-computing.com:389/O=Computing?CN" - URL to your LDAP server. In this case we are looking from the top of the directory tree (O=Computing) and are using common name (CN) to authenticate. The other parameters that Declan has in his example are all defaults, so I didn't use them.
Require valid-user - means that any user that successfully authenticates will have access.

Save the file and restart the Apache:
# service httpd restart
Stopping httpd:                                               [  OK  ]
Starting httpd:                                               [  OK  ]

Now try to log in. The Apache should ask for user name and password. Log in as you would in Notes client or iNotes.

And that's all! We now have Mercurial server running on a Red Hat server, with LDAP authentication, waiting for XPages stuff :-)

Let me know if something is not clear or you encounter problems. I'll do my best to help you.

Permalink | Leave a comment  »

It's been quite a while since the last post, but we haven't been sitting with our arms crossed. Far from it - we have actually successfully finished our first sprint! More about that in the next post.

The topic of today's post is requirement management. For me, a requirement has been synonymous with a short, unambiguous, comprehensible sentence describing a (sometimes very limited) set of functionality. Each requirement has to be testable, verifiable and traceable both up and down in the requirement hierarchy. In large projects, the requirements are collected in dozens of requirement specifications documents - for hardware, for software, for interfaces and all of that repeated on various abstraction levels. It is not unusual that there are hundreds or thousands of requirements describing a single product. System analysts then browse through all these documents (or a requirement management application if they are lucky to have one), trying to make sure that requirements really describe system that customer ordered, that there are no conflicting requirements, that test cases have been assigned to all requirements. A very difficult and demanding task and the one that often has direct influence on project's outcome.

Enter the world of agile software development. And forget most of what you know about requirement management. In most of the agile development, requirements take on a different form. Requirements are longer and often describe large sets of functionality. In fact, they are not even called requirements but user stories. User stories describe what customer wants and do so in terms that customer understands. It is the customer that writes the stories and it is the customer that writes how to test the stories. User stories are written in everyday language and are hence more prone to ambiguity and vague formulations. How can they then replace "traditional" requirements?

The answer is simple and lies at the core of the whole agile development process - communication, communication, communication. The user stories are not meant to be truth carved in stone (which is often the case with traditional requirements and changing them requires starting a massive change management process). Instead they are meant to provide a topic of discussion between the customer and the developers. During the course of this discussion, the customer's needs become more evident, more precise and easier to transform into the language that developers understand.

This discussion occurs immediately before the functions described by the stories are implemented. In the case of Scrum, the discussion happens in what is called Sprint Planning meeting, usually once a month. During this meeting, the team of developers, together with the customer representative choose what stories to implement based on the customer's business priorities. The functions described in these stories are then discussed over and over again, until there is a mutual agreement and understanding about what the story represents. The details that emerge during the discussion are sometimes converted into new stories, but more often they are simply remembered by all involved. Scrum is not about documenting what is clearly evident and understood. After all, the stories are to be implemented, tested and demonstrated in one month (or less) and everybody should be able to remember what was said for such short period.

When we do first requirement analysis and user story gathering, we use paper note cards that we fill-in by hand (template attached). The user story gathering is most often done by the customer representative (Product Owner in Scrum terms) and the team leader (Scrum Master). On the front side, they fill descriptive title, description and importance. On the back side, they fill how the story is tested. One example is given below:

As you can see, we are trying to use the "As a _____, I want to ______, because ______." form as much as possible, because we found out that it helped us to write good stories. Notice that this card has additional fields, such as ID, Link to and Story points, but at the time of initial story gathering, they are rarely used.

After the user stories are identified and gathered, the Scrum Master will transfer them into electronic form. We use simple spreadsheet for this. This spreadsheet is essentially Product Backlog that describes everything that is going to be developed. Once in the spreadsheet, the stories continue to live. They are constantly read, discussed and rewritten. The product owner moves them up and down, thus indicating their importance. When the time comes for Sprint Planning meeting, we print the stories directly from the spreadsheet and use them for further discussion and estimation.

And, based on what we saw in our first sprint, this technique works very well. During our sprint planning meeting we focused only on top priority stories. We estimated how many stories we could finish and made sure that we understood what the customer wants, but also that customer understood what she would get.

If you want to learn more about user stories, I can warmly recommend User Stories Applied by Mike Cohn. Another great resource that covers practical implementation of Scrum is Scrum and XP from the Trenches by Henrik Kniberg. Lastly, you can download Excel, OpenOffice and Google Docs versions of the product backlog template from Henrik Kniberg's blog.

Coming next: Our first sprint.

See the full gallery on Posterous

User_Story_Template.pdf Download this file

Permalink | Leave a comment  »

Let's see what tools we need for developing our application.

Development environment

First, let's take a quick look at how our development environment looked until recently. We were mainly developing for the Notes client, so 99.99% of the development work was done in the Domino Designer.

On the lifecycle management front, we were using Teamstudio CIAO!  for source control and our own, in-house developed, applications for bug tracking and release management. These applications were nicely integrated with CIAO! and the whole environment worked well. We had one development server where development and testing was done and we deployed finished application templates from this server to our production servers.

As I said before, application development is not our core business and in these financially difficult times we were - unfortunately - forced to abandon CIAO!. It was time to re-think our development environment. And the timing was right - IBM announced support for source control starting in the Domino Designer 8.5.2 (which improved in 8.5.3) and the slow but seemingly inevitable switch to XPages and Java made it easier to use well established source control systems such as CVS, Mercurial and Git (to name but a few).

So, we had to choose what source / version control tool to use instead of CIAO! We evaluated these:

1. SVN, a representative of "classic" check-out / check-in tools.  In my previous job, we used Continuus (which had been acquired by Telelogic, which was then acquired by IBM and incorporated in the IBM Rational family), not very friendly tool to use and difficult to manage. I must admit that SVN is light years from Continuus, the server is extremely easy to set-up (use uberSVN and it is as close to one-click setup as you'll ever get) and the integration with Domino Designer client is great (as demonstrated by Niklas Heidloff);
2. Mercurial, a representative of the new kind of distributed version control tools. For someone coming from the rigid world of check-out / check-in tools, it took me some time to bend my mind around it ("There is no spoon."), but once that was done the rest was easy. The Mercurial server is not as easy to install and configure as SVN, but is not an impossible task. If you need help, Declan Lynch has a tutorial for Ubuntu and if you need help with setting up Mercurial on Red Hat / CentOS, let me know and I'll help you.
   Unfortunately, I couldn't make MercurialEclipse plug-in (client connector) to work in Domino Designer. The latest version of the plug-in won't even show up in the Designer. The older version do show up and can be configured, but as soon as one tries to use it, the connector fails. I tried all possible combinations with no success.
3. IBM Rational Team Concert (RTC) is a massive tool used for managing complete development (planning, following, tracking, automatic software builds, source control) and represents only one third of the complete suite for Collaborative Lifecycle Management (there are also requirement management and testing / verification modules). We spent the least time with this one. The source control tool is somewhere between SVN and Mercurial, but we couldn't understand it quite well in the little time we had. We also failed completely trying to install the RTC Eclipse plug-in in Domino Designer. It seems that the gap between Eclipse and Domino Designer is too big for plug-in installation. You might still want to check this tool, however, as it offers very much functionality and it comes free for up to 10 developers.

So, what would it be? SVN or Mercurial? Mercurial won. There is no specific reason, as we haven't used any of them long enough to be able to judge them fairly.  At the end, there was such a trivial thing as a great tutorial to tip the scales in Mercurial's favor. Here's the tutorial (one of the best I've ever read); I warmly recommend it to anyone interested in Mercurial or facing the same decisions as we did: Hg Init.

But, what about the (non-existing) connectors? Well, we decided to use TortoiseHg desktop connectors. The difference between using TortoiseHg and MercurialEclipse is not that big, anyway. In Eclipse, If you use MercurialEclipse and want to work with Mercurial repositories, then you need to switch to a special Team Synchronization perspective. When I want to work with Mercurial repositories in the Domino Designer I simply start TortoiseHg Workbench. Either way, you only need one mouse click to switch your work context completely, from coding to synchronizing.
TortoiseHg is fast, stable and can do everything that MercurialEclipse can do. We configured the Domino Designer to automatically keep the on-disk project in sync with the nsf. That way we have one step less to perform. And because of the way Mercurial works, one doesn't need to worry that untested changes would break the build, as one always commit to local repository first.. If there is any interest, I could write a more detailed description of our Domino Designer / Mercurial setup.

We are keeping the rest of the lifecycle management infrastructure as-is. We won't have the same tight integration as before, but we'll manage.

One of the most important requirements that customer has for the future application is that it must be very user friendly and very fast and efficient to use. Which means that it will be a lot of changes and refinements of the application GUI. With that in mind we decided to invest in a GUI sketching tool and we opted for Balsamiq Mockups. Fast, easy and fun to use, it already proved its worth.

Before I wrap-up this section on our development environment, let me shortly address the Domino Designer. We are going to perform the major part of coding and developing using the Domino Designer. But, in case that we choose XPages path (No, we still haven't decided - there is some chance, albeit slim, that we shall choose traditional development for the Notes client) we are thinking about doing as much as possible of Java coding and development in the Eclipse client and then using finished libraries in the Designer. That way we can take advantage of all the benefits that the Eclipse offers for development, testing and deployment. Has anybody done anything similar? Any thoughts?

Documentation

This one is easy - Google Docs. Google Docs make it very easy for us to collaborate and share documents with our customer. They also provide implicit version control and track authors and changes. What more could one ask for?

In those cases when we need more robust editors or explicit version control, we are going to use IBM Lotus Symphony and Quickr repositories.

Project Management

We are using Scrum as project management framework, so we need some tools for planning, following and managing. We have evaluated a couple of them.

One of the is the already mentioned IBM Rational Team Concert. The RTC provides support for creating release and sprint plans,  product and sprint backlogs, progress charts, user stories and epics, impediments and some other useful artifacts. But, these functions come at the cost - the overhead of learning, setting up and managing the tool is rather large. Scrum is all about transparency, simplicity and manageability - and dividing work in small enough chunks so that no special tools and extensive documentation are needed to manage it. In that respect, having to learn a complicated tool to manage it felt a bit wrong. Furthermore, despite all of the features, the tool felt too rigid at times, not allowing us to do things the way that we wanted.

Another tool that we evaluated was the Rally Community Edition. This is a web-based tool and it is easier to use and faster to get-going than the RTC, while providing the same (Scrum) functions and features. And like the RTC it is free for up to 10 developers, but it is limited to a single project (no such restriction for the RTC).
And, even though the Rally was easier to use that the RTC, it still felt awkward to have to go through the click-here-fill-text-click-there routine for every little thing.

So, after reading a bit more and looking at some other tools, we decided that we actually don't need any special tool to support Scrum - we need a spreadsheet on Google Docs for our product backlog and whiteboard to keep track of sprint backlog and related tasks. It is as simple as that.

And that brings us to the end of this part. To sum it up, these are the tools that we are going to use: Domino Designer for coding (maybe with some help from Eclipse), Mercurial and TortoiseHg for source version control, Balsamiq for GUI mockups, our own applications for bug tracking and release management, Google Docs for documentation, sharing and collaboration, and once again Google Docs for Scrum planning and managing.

Coming next: Requirement management the Scrum way.

Permalink | Leave a comment  »

Why do we need methodology? The aim of methodology is to provide us with means - methods, processes, procedures and tools - to accomplish a goal. If we talk about software development, then the goal is to deliver applications that fulfil customers' requirements, on time and within budget. And to do that consistently, release after release, application after application.
Now, the problem with software development is that it is such a fluid and vaguely defined problem space. There are so many unknowns, so many changes that keep popping up throughout the development cycle that we can never truly grasp and fully understand either the problem or the possible solutions. The software development is, in many ways, what Herbert Simon would call an ill structured problem. Just think about it - we have ever changing and sometimes conflicting customer requirements, we have technology that improves at the pace that is impossible to keep up with, we have human factor to deal with. All of this means that it is more than necessary to introduce some kind of method in the process of software development.

As I already said, my background is in defence industry. Defence industry is traditionally very conservative when it comes to product development. We used the V-model of development, which is a variant of the waterfall model. In this model, every development activity (analysis, design, testing) is clearly separated and done in sequence. The progress of every activity is reviewed at predefined milestones (such as Preliminary Design Review or Critical Design Review) and the result is documented in lengthy written reports. Then, the next-in-chain team takes over.

This methodology is suited for big projects where very strict and formal project management is required. Clearly, majority of software projects do not fall in this category. That is why we saw a number of methodologies introduced in the late 1990s, with the single goal of making software development easier and more successful. Some of these methodologies can be put in the Agile category and they share the twelve principles of Agile software  as their foundation.

One of these methodologies is Scrum and we are going to use it to guide us in developing our application. Why Scrum? To give a short answer - because it best fills our needs. This is the methodology that we can best implement, having in mind our very limited resources - and I am aware that we still won't be able to realize all of the benefits. For that we would require a larger (around 7 persons), cross-functional team with analysts, GUI specialists, programmers, testers.

The Scrum is also diametrically opposite to the traditional methodologies, such as the V-model that I used before. Gone are the traditional requirements, design documents are deprecated, everything is done with as little documentation as possible and we - the developers - are supposed to manage ourselves. If you are interested in Scrum, I encourage you to read the Scrum Guide. It is a short (17 pages) document that is essentially a complete definition of Scrum. The Scrum Guide says that Scrum is simple to understand, but extremely difficult to master.

I also want to clarify one thing - neither Scrum nor the V-model are methodologies in the true meaning of the word. Neither prescribes tools or procedures for solving problems at hand. Instead, they should be more correctly described as frameworks. It is up to each and every team / project to decide how to solve, organize and manage everyday activities. For example, we used MIL-STD-490A and MIL-STD-498 to flesh out the V-model. In the following posts I will write more about our implementation of Scrum and the progress we make.

Coming next: Tools.

Permalink | Leave a comment  »

It seems that LotusScript Symphony API is broken in Notes 8.5.3. Our applications that use it (and that have worked fine ever since the API was introduced) stopped working and are popping out the error message that the USE or USELX module *symphonycfglsx cannot be loaded:

Those applications that use the UNO API work as they should. I've seen that some others have reported this issue as well. Anybody else seeing this?

And, does anybody know which version of Symphony is in the 8.5.3? It would seem that it is 3.0.0:

I tried installing Symphony FP3 yesterday, but it fails saying that it cannot find compatible version and points out that current version is 8.5.3. I think that the FP3 fix pack installer (for Windows anyway) only works with Notes 8.5.2. Maybe it is possible to manually update the installation, but I haven't tried that.

Permalink | Leave a comment  »

Almost a year ago, I wrote that Domino Collaboration Express users were not able to download Symphony installers from the Passport Advantage site.

It turned out to be a problem with licensing, and IBM promised to fix it by the time 8.5.3 ships. Well, today is that day and I am happy to say that IBM kept the promise: installers for both stand-alone and embedded versions of Symphony are now available for download for Domino Collaboration (and I believe Messaging as well) Express users.

Permalink | Leave a comment  »

So, we are going to build an application. Exciting!
First, let me provide you with some background information. It will help you to better understand the challenges we are facing and decisions that we make.

The customer
The customer is a crossover between a fitness and a wellness centre. We have had a good professional relations with them for a number of years. They are demanding, but they understand that IT has its own rules and are willing to learn and listen to somebody who knows more about it. They know about our experience (i.e. the lack of it) but they are confident that we'll be able to build their application. With that in mind, we agreed upon a very generous time plan that should allow us to breath freely, but which also guarantees that they will have an application before Hell freezes over.

The application
The application is a client tracking application. Something like a CRM, but heavily customized according to their needs. It should contain a list of their clients with general data, but also some specific items, like height, weight and calory intake. For every client a complete history of contracts, payments, treatments, exercises, measurements is to be kept with possibility to do various reports and analyses. The customer already has an application for this purpose, but it is not being used as it is not user friendly and flexible enough. So, they currently use pre-printed forms that are filled-in by hand. A very fast and convenient way, but they are starting to have problems with physical archiving of all the paper. Also, finding information from some time ago is often an impossible task.
The most important requirement that the customer has is that the new application must be simple and fast to use. To enter information in the application should be as fast and simple as writing it on the paper, or as close to it as possible.

The development team
Well, there is no development team. As I already wrote, application development is not our core business. It is only one colleague and I that are going to work part-time on the application. So maybe (and just maybe), I could count that there will be equivalent of one full-time developer working on the application. And that's not much. That means that we can't do much in a given period of time. But what worries me more is that we may not have enough knowledge and experience between ourselves to pull this through. I will try to mitigate this risk through (creative) requirements management and methodology support. Another risk looking farther down the road is application support. I am afraid that once we "finish" (I am aware that there is no such thing as finished application) the application, the management is going to move us to other duties, with no one left to support the application. That is something that I will have to deal with and try to persuade the management that we can make money on application development.

And that was the general background. Coming next: Methodology.

Permalink | Leave a comment  »

I am not an application developer. And I am not an administrator. I am electronics engineer who by chance ended up with an IBM Business Partner and learned to know Lotus Notes / Domino. And I liked it - stable, reliable, easy to upgrade from administrative point of view. And easy to use as a development platform. Security, deployment, all those difficult to master things were taken care of by the platform. I still remember my first programming experiences with Domino Document Manager. And the joy when I successfully connected a sales tracking application developed in Lotus Workflow  with document libraries in the Domino Document Manager.

Lot of time has passed since, and I have become a better developer. I made several business applications that we use in our company. One of those applications is even used by external customers in their own business environment. But, application development is not our core business and development of those applications was ad-hoc, without elaborate development process. No analysis, no design, no testing or documentation. Don't get me wrong: I've done all those things, but in a limited, unstructured and inconsistent way.
And since my background is electronics system development in defence industry, that kind of disorder was not to my liking. Back then, every step, every move, every thought that we made was according to some rule, standard, methodology or procedure. And everything was meticulously documented.

So, I have always wanted to implement a proper application development ecosystem for Lotus Notes/Domino in our company. And to develop applications using it. Lately, things have started to move in that direction. We were approached by a client who wanted a specific business application. We accepted the job and we understand that we must do it properly if we are to succeed.

In a series of articles to follow this one, I will be writing about the upcoming application and our efforts to develop it properly and "by the book". Whatever that book might be :-)

Permalink | Leave a comment  »

This has been around for a week or so, but since I wrote about the problem, then it is only fair to write about the solution as well.

So, if you experience problems with iNotes after upgrading to 8.5.2 FP3, head over to IBM support site, download and install the fix:

https://www-304.ibm.com/support/docview.wss?uid=swg21508250

We've done that and it works fine for us.

Permalink | Leave a comment  »

Well, this is definitely my favorite so far: Podvrške beach, island of Murter, Crotia.

Permalink | Leave a comment  »

I've got iNotes working on our server.

Our environment:
RHEL 5.6 32-bit (VMware virtual machine), English locale
Domino 8.5.2 upgraded to FP2 and then to FP3, English

We are using Daniel Nashed's start/stop scripts for Domino and there we set Domino locale to bs_BA.UTF8.
When I comment out the locale setting (i.e. Domino works with the default English locale), the iNotes works fine (including in Firefox 4).
If I set any other locale (e.g. de_DE), iNotes stops working.

So, it seems that this is something that affects Domino servers running on a locale other than English. If you can, change the locale of your Domino server to English and see if that solves it.

Permalink | Leave a comment  »

At first, it seemed that this could be a problem with Linux servers,  but users on other platforms have also reported the same issue.

So, it is platform independent, some are affected and some are not. While we try to figure out why this happens, we restored Forms85.nsf to previous version in order to get iNotes working again.

Forms85.nsf is located in DominoData\iNotes folder. You can find previous version of the Forms85.nsf in the DominoData\iNotes\852FP2 folder (if you don't have 852FP2, you might have 852FP1 or 85020 folders).

What you loose by going this way is the newly added support for Firefox 4 (among other things), but the iNotes will at least work. If Firefox 4 support is important, there is Interim Fix 401.8HF6 for iNotes 8.5.2 Fix Pack 2 or you can install Remote XUL Manager and add your domain to the white list (I tried both and they worked fine).

Permalink | Leave a comment  »

We have just upgraded our Domino servers to FP3 and the iNotes is broken. This is what you see when you try to access mail:

The server is Red Hat Enterprise Linux 5.6, Domino server was 8.5.2 FP2 prior to upgrade.

The same behavior is encountered for both Full and Lite modes, but the ultra-light mode works fine.

We checked with Firefox 4 and Internet Explorer 9 and both displayed the same problem.

Others are also reporting the same problem, so be aware of this.

Permalink | Leave a comment  »

IBM has released a white paper describing test results of a Lotus Domino 8.5.1 environment running on VMware vSphere 4. The white paper is quite exhaustive and covers some interesting scenarios.

Highlights:

• you can run Domino server with 4000 average mail users in a virtual machine with 1 vCPU and 8 GB RAM,
• even though Nehalem CPUs are faster, you'll do very well with older CPUs as well,
• disk configuration is important (critical), just like in physical systems,
• if your disk subsystem is slow, add more RAM - it really helps,
• if your users have large mail files, add more RAM,
• HTTP is much more demanding than NRPC.

We have been running our Domino servers on the VMware plaform for several years now and never had any problem. We are very small though, and it is good to see that Domino can be successfully virtualized on VMware even for larger environments.

Permalink | Leave a comment  »

Every so often, a Notes application needs to access external data. While the data may reside in virtually any external repository, in this post we'll only consider other Notes applications (or databases as they were often called prior to Notes 8.0).

Reasons for accessing other Notes applications may be various, but it most often comes to manageability and flexibility. Instead of developing one giant application that does everything a company or a department needs, we split the required functions in smaller chunks that are easier to develop, maintain, manage, support and use.

Let's take the notion of customer as an example. Many business processes need to know about the customers, some need to update existing or create new customer records. These business processes are supported by specific line-of-business applications - different sales tracking or help desk or project management applications, to name but a few.

Now imagine how inefficient it would be if each and every of these application were to keep its own customer records. Poor application users - they would need to input the same customer data multiple time into various systems and applications. And since that would be a job done by different users, we would inevitably end up with errors: in one application we would have IBM, in another I.B.M. while in the third International Business Machines. We humans easily understand that this is the same customer, but an application trying to correlate data from these different systems will see three different customers.

What we usually do to rectify this situation is to make an application to hold common customer details: customer name, address, contact persons etc. And we make those details available to other applications. We enter details once, store and share them with all that need them.

Nothing new or groundbreaking there, just plain common sense. It has been done numerous times before.

So, how do Notes applications share data? How do they communicate?

We who develop on Domino platform are very lucky. Domino has many functions that make data sharing between applications easy - we have very robust security that keeps data safe and we have flexible communication mechanisms that make it easy to access data - whether it is on the same server or on the other side of the Earth. And we have programming features in Formula language and LotusScript that make developing such data-sharing applications easy.

In vast majority of cases, we develop just as if the data was in the same database – same calls, same methods, same principles. We simply point Domino server to another database. A couple of common examples:

• Use @DbColumn to retrieve contents of a view column.
  This is often used to populate choices in a dialog list. For example, you could use it to access application with customer details and get a list of customer names. The names are then displayed as a dialog list. All that is required to get data from another application is  formula similar to this:

  @DbColumn("";"Server/Acme":"customers.nsf";"CustomersByName";1)

• Use LotusScript to create document in another application.
  Imagine that a customer wants an entry created in log application each time a support request is saved in Help Desk application. You could put code similar to this in the PostSave event of the support request form in the Help Desk application:

  Dim w As New NotesUIWorkspace
  Dim s As New NotesSession
  Dim LogDb As NotesDatabase
  Dim ThisUIDoc As NotesUIDocument
  Dim LogDoc As NotesDocument

  Set ThisUIDoc = w.CurrentDocument
  Set LogDb = s.GetDatabase("Server/Acme", "log.nsf")
  Set LogDoc = LogDb.CreateDocument

  Call LogDoc.ReplaceItemValue("Form", "log")
  Call LogDoc.ReplaceItemValue("Comment", ThisUIDoc.FieldGetText("RequestDescription"))
  Call LogDoc.ReplaceItemValue("LoggedBy", s.UserName)
  Call LogDoc.ReplaceItemValue("LoggedOn", Now())
  Call LogDoc.Save(True, True)

These two, as well as many other similar, approaches are regularly used in Notes applications. And they work well (please do see Disclaimer at the bottom).

But, can you spot a problem with both of them? Yes, it is those hard-coded values all over the place.

In a production application you would make it possible to define server name and database location, probably in a profile document. Then you could do something like this:

Server := @GetProfileField("Settings"; "DBServer");
DB := @GetProfileField("Settings"; "DBPath");
@DbColumn("";"Server":"DB";"CustomersByName";1)

This way, at least, you wouldn't need to change code each time your target database moves. But what about those other values? They pose two different problems:

• First, you would need to know names of design elements from the target database which you are going to use: forms, views, fields. You would also need to know what columns a view has and what their content is. Usually, this is not big deal.
• Second, what if something in the target database changes? What if some field changes name or a column is deleted from a view?

If you are in charge of both applications then you can try to find a middle way that will allow you to make necessary changes in one database without causing too much changes in the other(s). But if you are not, and the target database changes in a major way, then you may be looking at some quite extensive code re-write. If you take a look at some of IBM templates that ship with Lotus Notes, you will probably find some strange looking design decisions: they are there for backward compatibility. However, the vendor from whom you purchased your target application may not be that kind, forcing you to re-write your code.

What can be done about that?

I was thinking about this lately and thought how it would be nice if we could have something like public methods in OO programming or stored procedures in SQL for our own Notes applications.

We would expose (i.e. made public) methods (functions, subroutines) for other applications to use. These methods would take a list of arguments and return values or perform some action in target database.

Wouldn't it be nice if you could, for example, call getCustomerNames() method in a customer application and get back a list with all customer names. Or if you could call createLogDoc(ThisUIDoc.FieldGetText("RequestDescription"), s.UserName) to create a log entry in the log application.

See where it goes? If you have any experience in OO programming, the concepts will be very familiar. It goes to hide implementation details from the caller by publishing a public, non-changing, interface.

Now, I no longer need to know the view name and the fact that customer names are stored in column 1 in order to get list of customer names. Nor do I need to know form and fields names in order to save a log entry. The implementation details could be changing on a daily basis. And I wouldn't care as long as the call (interface) remained the same.

But, how do you do that in a Notes application? As far as I can tell, the only way is to use web services. Web services were “invented” to make possible exactly that we are looking for: to allow applications to talk to each other and exchange data using standard “language”. The first application does not need to know anything about the second: on what OS it runs, in what programming language it was written or how does it implement requested functions. The only thing that matters is that they both speak the same “web services” language and are aware of functions that each other can perform.

Web services were introduced in Lotus Domino in version 7. I recommend you to read the two-part (part 1, part 2) article written by Julian Robichaux over there at developerWorks. It will give you a nice foundation for further work with web services.

In the next post, I will show how we can use web services to make communication between Notes applications more robust and future-proof by using web services. When web services first appeared in version 7, the Designer only had support for web service providers. Fortunately, that has changed and I will show you how easy it is to create both web service providers and consumers in Domino 8.5.x.

Disclaimer: All code shown in this post if for illustration only. It would need to be made more robust for production use. At the very least, you would need to implement error handling and do some performance optimisations. If you need help in doing that, let me know and I will be more than happy to help you.

Permalink | Leave a comment  »