RooJSolutions

JSON based calendar provider for thunderbird and lightning

2008-10-01 20:15:00

Article originally from rooJSolutions blog
For this weeks hack, I've been looking at Lightning, the calendaring extension to thunderbird.

What brought this on was we where discussing a bug tracker for the Anti-spam / Mail management software, and none of the software that i've seen for this either is elegantly written, easy to set up or simple to use.

To me the 'ideal' way to handle this, having played around with Lightning, is to have a calender provider (that's shared) and have all the key people able to drag a 'requirement email' into the TODO task list, or similar. Where people can submit bug's by signing up to a calendar/todo list and just add it as a task.

Obviously not suited to all types of projects..., but I think you get the jist that using lightning to edit these bugs would be pretty compelling due to it's integration with your primary mailer.

State of the current providers.

So to see if this could work, I had a look at the current list of network providers available. ICAL, CalDav, Sun's Protocol, and there's a google provider.

On the face of things it looks like that's quite a choice, but if you look closely at all of these, one thing stands out like a sore thumb. The backend server to deal with all these protocols is an absolute nightmare to create, or modify to suit other types of applications (think of a shipping managment application that we could integrate order delivery information with the calendar, so you could add notes about the delivery etc. via the calendar item.).

Part of the core problem with the existing protocols can be summed up with one dirty word "XML"! - CalDav and google being the worst of this. If you look at the problem today and tried to solve it, the first thing that would come to mind would be JSON/REST. It's the perfect set of protocols to ensure that backend server would be trivial to write, and the front end would not be such a mess. (especially as Lightning's written mostly in Javascript anyway).

Hacking a provider

There are really only two examples I could find of calendar providers for lightning. Google is one, and there is also an extension called hcalendar - that pulls calendar's in from web pages (as far as I could see). The second of these two proved to be the goldmine for creating a JSON calendar provider extension.

hcalendar provides most of the code needed to implement any kind of provider, it only implements the getItems() method. However it has stub's for all the others.

You can browse through the code here, calJSONCalendar.js, I did run into a few issues with Async requests though, as It looks like you need to implement a request object (as per caldav and google calendar), otherwise you run into problems with transactions on updating. So as a simple workaround, the extension uses sync requests for updates and adding.

How it works

getItems() - get a list of calender or todo items.

is a GET request, with params
- aCount = max number of resulsts
- aRangeStart = date start ISO format
- aRangeEnd = date end ISO format
- caltype = calendar data type (C = calendar, T= todo)
- filter (not yet)
eg.
http://your_calendar?caltype=C&aCount=0&aRangeStart=2008-01-01....

this returns a json array something like this.

[
   {
	id: 1,                      // your data base id (ignored)
	uid: "XXXXXXXXXXXXXXXXX",
        dtstart: "2008-10-10 12:12:12",
        dtend: "2008-10-10 12:12:12",
	caltype : 'C',
	ical : "  <<< Calender data in ical format >>" 
	
	// all the other data is optional (if you do not have a ical record, it can build 
        // most of the event data from properties

	privacy: "default";
	created: "2008.....",
	last_modified: "2008....."
   }, 
   ......
]

modifyItem() / addItem()

is a POST request, with the params pretty much the same as the json data above (with the ical record in the ical POST entry) - the resulting entry is returned by the call in the same format as above. At present ical is the main thing that is posted, however I think it needs to send quite a bit of the generic information.

deleteItem()

is a POST request, with the property _delete_={uid},
(currently we ignore the response) - it should probably be
{ success: true|false, msgid : 1, msg: "A string error" }

the body of this message contains a example server implementation for the extension. - you can download the extension here: jsoncalendar-0.2.xpi, (REQUIRES LIGHTNING 0.9!!) or have a look at my svn repo JsonCalendarProvider for how to develop test it.

Read on for the SERVER BACKEND CODE...

local caching and JSON bugs.

2006-12-22 13:00:00

Article originally from rooJSolutions blog
One of the projects I'm working on is using XUL to render data, we started testing a few weeks ago, and discovered that the load times/ response where quite bad. It was pulling too much data from the server, and the application would either hang (if we didn't use asyncrounous xmlhttprequest calls), or we would have to show a slidy to indicate that data was being loaded. Neither where ideal.

In discussing the application with the client, they also requested that the application remember some of the states (eg. the data filters). Normally you might consider cookies or having a preferences database, but I remember seeing something about firefox2 having localized session storage.

You can read the session storage specification here, It allows you to store strings totalling upto about 5Mb on the client side. However, the reality is that you really need to be able to store javascript objects or similar in it. So since we where using JSON to transfer data, I thought 'how about serializing the data back into JSON, and storing it in the session.'

I started off by using the JSON js code, however this code is unusable for 1 major reason - it adds a method to array's/objects, resulting in the fact that foreach() always returns an additional member. (breaking almost all the existing javascript code!) - I also ran into problems with recursion in firefox (but I think thats due to me forgetting to add the String.prototype method.)

So in the extended entry is a quick kludge (without full unicode support) for converting items into JSON in javascript, so that something like this works:

sessionStorage.setItem("mycache_somedata", JSON_toString(mydata));

... and onloading..

var mydata = new Array();
if (sessionStorage.getItem("mycache_somedata")) {
   eval("mydata = " +  sessionStorage.getItem("mycache_somedata").value);
}

SVG/XUL Outlook style Calendar

2006-02-04 09:23:56

Article originally from rooJSolutions blog
A few weeks ago, a friend sent me a link to zimbra , While I quite liked the look of their calendar, i was not too keen on the java stuff. The XUL based Calendar I wrote a while back, while getting heavily used, (I basically track all my work in it). had not been developed any further, mainly due to the Proof of concept style hack that the code had ended up as.

And as I had recently upgraded most of my browsers to Firefox 1.5, and almost got my name in the mozilla hall of fame for fixing a tiny bug in it;). I was a little intreged by the SVG (and canvas) that comes built into it.

So in a quiet afternoon, I sat down and started hacking away to see how easy it would be to write a calendar application using these new graphic libraries.

You have to click on more, for a detailed breakdown of what is involved in interactive SVG...

Why XUL will win..

2005-08-26 23:33:47

Article originally from rooJSolutions blog
My favourite blog post this year has to have been Jamie Zawinski's "why working on hula would just not get you laid."

The obvious counterpoint to this is, well what would get you laid (or at least let's you say, god, this is just sooo cool, that it's fun). Well XUL just does this every time for me. Clients absolutely adore it, (it looks so similar to their normal desktop applications). As a developer, it is the perfect opposite of Microsoft products.

predictable
reliable
well documented
portable.

It works everywhere, and doesnt involve huge kludges to workaround known bugs in different browsers. real "write once, run everywhere."

Just to counterpoint that introduction, XUL is so dangerously cool, it even makes writing a mini groupware project fun.

I've been using Evolution quite a bit recently, but It's just not quite there for calendaring. I access my data from at least 3 machines, let alone on the road so there is no real clean way at present to do this). . It would also be very useful to keep track of what projects I'm working on, and for which client. So the hourly billed clients get something resembling the right bill.., and the fixed cost projects can be checked to see how badly I underquoted.... - now there's a specification!

So out of sheer fustration, last month I started hacking on a XUL calendar - doing the way it's supposed to be done (I consider sunbird as a rather poor example of this.). You can get a bit of an idea of it by just testing one of the templates - week.xul

Adding entries work by highlighting a time block, then pressing a key (or the new button), then filing in the details. This for me is what a calender should do, no fuss with popup windows or crap like that. (have a play with colours - that's really nice here, as changing projects in the real version set's the colour..)

Drag works reasonably ok for resize / move. There is associated month.xul and groupweek.xul (which are less developed), and the project/client/user admin stuff. But to be honest this is one of those - I need this, if someone else wants to play and send me patches, feel free, but I'm not going to get all excited about packaging and releasing it...

But that's the thing about XUL, you just feel like playing with it because it's fun (like todays addition of TODO to it), I cant remember the last time I had that feeling working around bugs with IE's javascript ;) [google for IE bugs if you are in doubt there]

Remote XUL Html Editor

2005-08-09 11:17:00

Article originally from rooJSolutions blog
After the last post, I finally managed to get to the point where the simple XUL HTML remote editor works.

Source code is available via Subversion
http://www.akbkhome.com/svn/XUL/HtmlEdit
Browse the source
Try it out.

Usage is pretty simple:

Install the files on your server

Add the css header

<?xml-stylesheet href="HtmlEdit.css" type="text/css"?>

use the htmledit tag, it uses attributes
eg.
```
 <htmledit value="hello world" />  
```

editorheight="400" (self explanitary)
editorwidht="700" (self explanitary)
imagepopup="name" where name is the id of a popup
linkpopup="name" where name is the id of a popup
value="the text to go in the editor"

For the image popup:

use imageurl="xxx" on the menuitem for the real image url to insert in the editor

For the link popup:

use url="xxx" on the menuitem for the url to insert into the editor
the url "pick:" just uses a javascript prompt to ask you for the url.

XUL and HTML editors

2005-08-04 10:11:01

Article originally from rooJSolutions blog
As part of the shop design, I've specified a HTML editor, similar to fckeditor or htmlarea to edit the product descriptions and information pages. However, this editor has to be embedded into a XUL interface....

My main website (not the blog bit), has an "edit this page" link at the bottom, which already uses a HTML editor inside a XUL interface, but the code behind it more a proof of concept, rather than a production toolkit. So for the shop I decided to look at cleaning this up and making it modular..

XUL has a great mechanism for adding widgets like this, with a single stylesheet include, and an XML file you can create your own custom widgets.

In the XUL interface file (your main page that you want to use the widget on), you add this line

<?xml-stylesheet href="HtmlEdit.css" type="text/css"?>

(where HtmlEdit.css is our css file for our widget.)

Then somewhere in the body, you add

 <htmledit value="hello world" />

(telling it to use your new custom widget.)

The CSS file is very simple..

htmledit 
{
            -moz-binding: url("HtmlEdit.xml#htmledit");
            max-height: 200px;
}

basically using the -moz-binding attribute to map the widget to an xml file (the #htmledit, pointing to a specific binding) - as the xml file could contain multiple widget binding (although I wouldnt recommend it)

Then your HtmlEdit.xml contains the <bindings> with <content> - the xul elements that make up your widget, and <implementation> with all the properties and methods that your widget has (with javascript for the methods) - if you use mozilla's lxr and search for "<bindings", it's easy to find examples.

Now all that stuff is quite simple ;) - what I had wanted was to turn my html editor into one of these.. Not quite so simple.. mozilla's design kind of make that less than easy..

Mozilla has an <editor> tag, this however does not work on remote xul.. - so we have to use html:iframe, and set
```
{iframeobject}.contentWindow.document.designMode = "on";
```
html:iframe's dont appear to be useable inside of <bindings>
even if you use the binding to append the html:iframe after it'self using DOM, you can not set the designMode, as the privaliges appear to prevent bindings doing stuff like that to the containing page.

In the end I suspect some kind of mix between a <script src="mylib.js"> and using the bindings to store the editor bar may be the best solution.. - But let's see how far I get ;)

[update] It turns out that it is feasible to put it all into a binding! - after a little hint on irc.mozilla.org#xul, I was able to create a HTMLIframeElement using importNode, and DomParser.