Updating existing Domino web applications in 2014

I was recently asked to update on old web application to improve the user experience and bring a "fresh and functional" new front end, making use of the latest development tools available in IBM Domino. This is a web application, not a website. The reason I make that statement is that the system revolves around processes, workflows, data and forms. It's a working environment and not simply a site that provides information. The system is BIG, with 20 odd databases, hundreds of forms and views, dozens of agents, subforms, shared fields, script libraries, pages etc... You get the idea. The development took around two years of incremental builds to reach the current release which has remained stable with small updates now for the last 4 or 5 years.

It was originally built in Notes/Domino 7 and uses many old style Domino web development techniques. These include pass-through HTML, Javascript and CSS libraries (attached or pasted into pages), html header fields, computed fields which generate HTML, Lotusscript agents performing background tasks and also generating reports and exports, subforms for common html and the list goes on. Now there are a lot of features in the current system that required a lot of coding, work arounds and hacks. For example the code to handle the view navigation was hand crafted using Javascript, CSS and AJAX. The structure of the forms was built using pass-through HTML, embedded standard design elements, CSS, code attached to individual fields, code attached to the form itself. Basically there's code everywhere!

So 8 years down the line the system is being used in a good number of companies, has been constantly supported and updated where needed and the bugs ironed out! The problem is the world has changed in that time and the system needs to move with those changes. As is the case with most systems built over 4 or 5 years ago, it was designed for installation on a server at a customer site. It wasn't designed with the idea that one day we would want to host multiple instances of this in a "cloud" environment for lots of companies or that we might want to throw out a lot of the UI and replace it in the future, however that is the case in both instances now!

So I set out my objective:

To provide a richer but simpler design by utilising the feature rich tools now available, removing as many hacks and as much surplus code as possible by using out of the box design components. But still make use of the investment already made in the product by not re-developing the entire system and re-using as much of the existing design as appropriate. The user interface needs to be familiar to people using sites such as Facebook and other social media. It should follow current trends in respect to layouts and design and if possible use standard CSS frameworks. Importantly it needs to provide mobile/tablet access to specific elements of the system if not the entire system.

So a few months ago I started to investigate what I could use to make this happen. Clearly in Domino using Xpages was the obvious choice as it provides many of the features I need out of the box. In addition I wanted to make the layouts standard and simple to maintain so the use of a responsive CSS framework such as Bootstrap or Foundation seemed like a plausible solution. But here lies the problem, I have little experience in Xpage development (although I understand quite a bit I've never used it in a project!) and I have never used either of these frameworks. Now that in itself isn't a big problem as long as you have the time available to learn the technology and best practices, something that can take months or years, time I simply do not have right now.

As I dug deeper into the work involved I realised that rather a lot of the design in the existing system could not simply be re-used. The existing page layouts currently in subforms could not be easily transferred into Xpages or Custom Controls due the use of pass-through HTML, computed fields, embedded elements and code in HTML Headers etc... Pretty much most of the existing code that generates front end would need to be re-developed if I want to use Xpages and/or a CSS Framework. I would have to create new Xpages for every form and view I wanted to keep and since just 1 of the 20+ databases has 97 forms and over 100 user views, that became an unrealistic proposition. I chatted with Paul Withers and also discussed some of the issues with the community on Twitter and it became apparent that even experienced Xpage developers are not tackling moving large scale systems to an exclusive Xpages design due to the effort required.

So the conclusion I have reached is that I should look at using Xpages to add functionality to the system but not replace existing functionality. For example the development of the mobile interface might well be done in Xpages using existing functionality found in the Extension Library (Thanks to the gang at OpenNTF who put that together!). The use of a CSS Framework and replacing existing forms/views with Xpages will have to be out on the back burner sadly, unless I can find a simple and quick way to do it.

Space Shuttle Atlantis - Final Voyage!

On May 14th 2010 the Space Shuttle Atlantis will roar into the heavens for what will most likely be her final flight... and I will be there to see her go!
I remember watching the first flight of the Shuttle, when I was 11 years old. The teachers has squashed the whole school into a classroom at school in England. A television was rolled out and switched on and we sat and watched...
I had always had an interest in space and NASA and seeing the Shuttle launch started a life long ambition to one day see one fly. 29 years later the dream is finally coming true (as long as a certain volcano doesn't spoil the party).
When I heard last year that the Shuttle was being retired at the end of this year I really didn't believe it would happen. I mean how could something as iconic as the Space Shuttle no longer be flying? But then of course I could say the same of Concorde. Both vehicles suffering from the same issue, funding. When it became clear this would be the last year I made a decision that I had to make a launch, no matter what the cost.
A couple of weeks ago I entered a draw being held by NASA. They are holding a special tweetup at Kennedy Space Centre for followers of @NASA and were selecting 150 people from all those who applied to attend the 2 day event. So I entered not expecting to be selected only to get an email confirming I had been picked! Not only would I be able to see the launch, it would be from the VIP/Press area, the closest point you can get during a launch. To add to that we would get a tour, meet astronauts and shuttle engineers and get a look behind the scenes, to see the cogs that make the space programme engine run. If that wasn't inspiration enough to go then nothing would be.
So in 5 days I will head off from Gatwick to Orlando for what I expect will be one of the most amazing sights I will ever get to see.

Server Time faster or slower than OS?

In the last few days our old mail server which is running Domino 8 started doing odd things with the dates and times. Mail which arrived at say 10:15am would show as arriving the previous day at 05:23am or some other random time. The amount of time the server was out from the OS date time was not consistent and would vary.

Having checked all the usual suspects such as regional settings, default user etc... I still could not find an obvious answer. Then by luck and a bit of cunning query writing I found a technote which described exactly what we were experiencing. Apparently the Domino server was suffering from "Time Creep", where there is some inconsistency on the OS ticks v the OS time and apparently was caused by a bug in the Domino Time package. This has been resolved in 8.02 and 8.5.

To resolve the issue there is a handy Notes.ini param you can set which tells Domino to sync the time with the OS:

SERVER_TIMESYNC_INTERVAL = 1 - every 5min (default)
SERVER_TIMESYNC_INTERVAL = 2 - every 1min
SERVER_TIMESYNC_INTERVAL = 3 - every 60min
SERVER_DISABLE_TIMESYNC=1 -disable polling

Having added this to the server it now appears to be correct but I will need to monitor it for a few days to be sure.

Here is a link to the full technote:
http://www-01.ibm.com/support/docview.wss?rs=899&uid=swg21327441

Xpages not loading? JVM errors? - Solution

If you read my last post you will recall that after upgrading my Dominio 7.02 server to 8.5.1 I could not get xpages to load from the server. I would get HTTP 500 errors in the browser and a http jvm: severe: clfad####e: error on the console.

I did the usual search on the net, tried the suggestions such as re-installing the server after removing the jvm directory, running a clean on the project from designer etc... But still no joy. Asked around a few fellow notes peeps and still no luck.

I tried to view two different xpage databases, the first was the standard 8.5 discussion and the second was a simple bespoke database with a single xpage containing a label and a field. Neither would load, however both would preview on the client.

After a few hours of investigation this morning and much hair pulling, desk head banging and caffeine consumption I have found the cause.

The error I was getting seems to be a fairly generic one:

HTTP JVM: SEVERE: CLFAD####E: Exception occurred servicing request for: /xpages/xpagestu.nsf/xp1.xsp - HTTP Code: 500
HTTP Web Server: Command Not Handled Exception [/xpages/xpagestu.nsf/xp1.xsp]

Unfortunately turning on the "Display Default Error Page" in the xpages tab of the database properties did not work and no additional information was displayed making tracking the problem down a bit of an issue.

So I turned on Http debugging using "tell http debug session on" from the console. This generates log files in the IBM_TECHNICAL_SUPPORT directory under the server data directory. After trying a few requests a had a look at the logs and the only error I could see related to a completely different database. The log file was showing an error with a cookie used in another web application on that server.

The error showing in the logs was :

java.lang.illegalArgumentException: Cookie name "some name" is a reserved tokenSo initially I ignored that and continued searching... After another half hour or so I went back to the log file and still the only error recorded was related to a cookie from another application. So I decided to clear all my cookies out and try again. Hey presto, it loaded!

The end result is that a malformed cookie name, in this case one that has a space in the name, stops xpages loading to the browser!

Now I need to do some more testing but what this means to me is that the other web application on our server, if used, will stop xpages working on a users browser. Now I don't know if there is a way around this without a a lot of changes to the other application. I will enquire from people at IBM etc... if there is a solution but at least I know the cause now.

Xpages not loading on Server, JVM errors

Finally last night I decided that it was time to upgrade our existing 7.02 server to 8.5.1 as I want to make use of xpages in our web applications. So I backed up the 7.02 directories and did an install of 8.5.1 over the existing directories.

This seemed to work ok, the server booted up OK after, was showing as 8.5.1 on console and all the expected tasks such as xsp etc... loaded on server start. The only problem is that the server will not serve up an xpages to the browser, all I get is an "HTTP error 500" in the browser and a "http jvm: severe: clfad####e: error in the notes log".

I've tried the 8.5 discussion database, a brand new database with a single xpage and just a label and field and neither will load. All preview OK from the client.

I have tried to re-install after deleting the jvm directory so it was recreated, still no joy. I have signed the databases with the server ID and that did not work. Regular http pages are server, i.e. forms, views, pages etc...

So I have an 8.5 server that won't load the very things I need to be using!

Any ideas, is there something else I need to have configured? Do you need to have an internet site document for xpages to work? I noticed in the server document there is nowhere to map javascript directories but there is in the web documents.

Dates and Times, how do you handle them?

So my last post on dates and times got me thinking, how is everyone else dealing with them for international type applications. I am now curious how people store, report on, use dates and times.

So if you had a "Global" application, where the users are in any timezone what would you...

1. Store in the documents
2. Use for views
3. Use for calculations
4. Include in emails
5. Deal with front end and server side use

Having exchange a few mails recently it seems that people do things differently. Some store in local, some convert to GMT/UTC, some store mulitple dates in different formats.

Please leave a comment!

Notes and Timezones!

I have a rather difficult issue to resolve, well in my mind it is, it may well not be! "Time Zones!"

I have a system that is a global application, it can be used via a browser or via a notes client The system stores lots of dates to track SLA's and the SLA's themselves are computed based on the day of the week and the time of day. Normally storing dates is fine, you use a date/time field and leave notes to handle the display. Unfortunately the SLA issue here causes me a problem.

So here is where it get's complicated, all the dates/times need to be in CET (Amsterdam) no matter where they are set.

1. The server is in UK running GMT
2. Users from the web will use local time on Client machine which could be any time zone but dates can potentially be calculated in WQS agent on server
3. Administrators can be anywhere but will use a Notes Client
4. Reporting must be in CET (Amsterdam)

So when a document is created via web or client, the SLA's need to be looked up based on the hour of day and day of week as I said. This must be based on CET (Amsterdam) no matter what the local time zone of the user is or server is.
Now I thought I had something working until I turned my local machines clock forward to April, which of course is GMT+1. Now my CET dates/times are still showing GMT +1 and not taking DST into account. I guess I need to know if the date/time in Amsterdam is DST even if the machine running the code is in New York for example.
So currently my codwould show 01/01/2010 19:00 GMT as 01/01/2010 20:00 CET but 01/04/2010 1900 GMT+1 shows as 01/04/2010 CET as DST is not being taken into account.
I have to admit this is driving me nuts!
So any advice? Ideas?