When I started this blog, I wanted to keep it limited to my private open source work and not write about the work that I do as part of my regular software engineering job at Intel. That distinction is mute now because as of January this year, my main job at Intel is about SyncEvolution and data synchronization in Moblin. That means I now have the chance to work on it not just in the evening and weekend, but also during the day. No wait, I mean, I can work on it during regular work hours and have my normal life back in the rest of the time…
This change is an opportunity for SyncEvolution to go beyond its traditional role of a simple SyncML client for Evolution. It’s too early to announce specific plans, but the intention is to include a GUI and also work on direct device-to-device and device-to-desktop synchronization. This will all be open source, so if you want to help, then let’s talk (see below).
Moving to Synthesis SyncML Engine
The other big change in direction is that SyncEvolution 0.9 will be based on the Synthesis SyncML Engine, which is getting re-released under the LGPL by Synthesis. The Funambol C++ library, the basis for SyncEvolution in previous releases, has served SyncEvolution well. SyncEvolution wouldn’t exist if Funambol hadn’t released their source code. I hope that my contributions to Funambol (port to Linux+autotools, interoperability testing, CPPUnit testing framework, to name just some of them) will continue to be useful even though now it is time to move on and I’m not going to contribute directly anymore.
I look forward to similarly productive cooperation with Synthesis. So far, I have added the autotools build scripts to it, made sure that it compiles cleanly with gcc on 32 and 64 bit, and fixed a few issues that came up due to the different way of using the source in SyncEvolution.
Why change a running system? It’s because the Synthesis code base is arguably a better match for SyncEvolution’s goals. The core engine is written in C++ and implements both the client *and* the server side of the synchronization. The Funambol server is implemented as a Java Enterprise application and depends on an application server like Tomcat. The bundle made available by Funambol of course runs on normal desktops, but this is not a replacement for a normal desktop application. Refactoring the server so that its SyncML engine can be used without an application server would be a lot of work and likely lead to patches that cannot be accepted upstream because they change the core architecture.
There are several other advantages that come from switching to the Synthesis engine:
- suspend/resume support (not used yet in 0.9 beta 1)
- description of SyncEvolution capabilities are generated automatically
- WBXML, the more efficient encoding of SyncML messages, is now supported (but disabled by default in 0.9 beta 1 for further testing)
- SyncEvolution can and will be licensed as LGPL, which is a better choice for a project that is meant to become the infrastructure that others build their applications upon.
Because the Synthesis engine implements the data format conversions and merging required for a SyncML server and makes that functionality available also to SyncML clients, it was easy to implement another fundamental change in SyncEvolution: 0.9 switches from an “all items are sent verbatim to the SyncML server” to a “parse and convert” data handling model. The argument for the former approach was that the SyncML server should be the only entity in the system which does data conversion. But faced with real-world constraints, the previous releases already had to deviate from this approach to accommodate for minor client/server incompatibilities and for vCard 2.1 support.
The main reason for going to full semantic conversion is vCalendar 1.0 support. Support by servers for iCalendar 2.0, the only format supported by 0.8.1, is often still incomplete or even non-existent. By doing the conversion on the client side, SyncEvolution is now able to synchronize events and tasks with a wider variety of servers.
It is still true that properties not supported by a server cannot be synchronized to other devices, so using a server with full iCalendar 2.0 support is recommended. But in contrast to 0.8.1, information that can only be stored locally is no longer lost when receiving an incomplete update from the SyncML server thanks to intelligent merging provided by the Synthesis engine. This depends on an accurate description of the server’s capabilities, which might not be provided by all of them – still needs to be tested in more detail.
What do these changes mean for related projects?
I look forward to continuing the cooperation with Funambol and ScheduleWorld on interoperability. The use of open standards for synchronization is going to benefit everyone involved because it allows users to synchronize who wouldn’t be able to do that otherwise, but it requires testing to ensure that the different components in the system really work well together. Other SyncML server developers and operators are welcome to join the SyncEvolution interoperability testing as long as they:
- provide test accounts
- assist with analyzing problems
- support SyncEvolution users who want to use the service
With the goal of supporting direct synchronization, SyncEvolution enters a space that other projects also focus on, for example OpenSync and Conduit. We already discussed opportunities for cooperation with OpenSync: OpenSync currently doesn’t have stable code for conversion between different formats like vCalendar and iCalendar or vCard 2.1<->3.0, which is essential for a useful 0.40 release. Perhaps the Synthesis code can fill this gap. The Conduit project seems to be less focused on PIM data and mobile devices, but if there is a chance to cooperate, then it’s certainly worth discussing that.
The goal is to continue with SyncEvolution and Synthesis not just as open source, but also as open projects, with as much communication on public channels as possible. This is just getting started, so bear with us (and kindly remind us!) while we figure out how to do this properly.
SyncEvolution 0.9 beta 1
The focus of this first beta was to reach the same level of functionality and stability as in
0.8.1. Therefore this beta does not yet bring much new features; this
will be the focus of further beta releases until finally 0.9 will be a
full replacement for 0.8.1.
- vCalendar 1.0 is now supported. Because this hasn’t been tested that much yet, events and tasks are still disabled in most of the default configurations (SF #2635973).
- Both libcurl and libsoup can be selected at compile time as HTTP(S) transport mechanism.
- SF #2101015: Expect: 100-continue header results in 417 Error with proxy. Should no longer occur with the HTTP transports in this release.
- SF #1874805: Syncronization with Funambol results in loosing all-day property. This now works thanks to the Synthesis data conversion rules.
- SF #2586600: Synchronization with mobical.net fails. Should work now because of the different SyncML implementation (untested).
- SF #2542968: separator for categories should not be escaped. Done correctly by the Synthesis vcard conversion.
Known shortcomings in this release which will be fixed before the final 0.9:
- Verbatim file backups of items on the SyncML server are currently not possible: the SyncEvolution “file” backend still exists, but all items are converted by the Synthesis engine and therefore must be in a format supported by the engine.
- HTTPS can be used with libsoup, but certificate checking is always disabled. Need to find a portable way to determine where the certificate file is on various systems.
- Log file handling is not yet unified: the traditional client.log contains only high-level SyncEvolution log entries. Low-level SyncML and engine log entries are in sysync_*.html files.
- stdout and stderr messages from system libraries are visible on the console. 0.8.1 used to redirect those into the client.log to hide this noise; this will be added again. In the meantime, ignore messages like “Deadlock potential – avoiding evil bug!”. This is liborbit telling us that it is (hopefully successfully) handling something nasty.
Download + Installation
The source of SyncEvolution 0.9 beta 1 with the corresponding Synthesis source included is available here: http://www.estamos.de/download/syncevolution/sources/
Precompiled binaries will be made available for Moblin soon. For other platforms please compile from source. Compilation, installation and usage works as described in the README and on the upstream project site. Here’s how one can compile and use the latest sources:
git clone git://git.moblin.org/syncevolution.git
../configure --disable-shared --with-synthesis-src=git://git.moblin.org/libsynthesis.git
The following mailing lists are used for discussions:
SyncML related issues can be filed there or directly on synthesis.ch.
The code is available in git repositories:
- upstream Synthesis:
- staging area for patches against upstream Synthesis:
The goal of the libsynthesis repository on moblin.org is to prepare patches developed by the core SyncEvolution team and publish them for upstream inclusion. If you want to contribute patches yourself, then please work with the Synthesis repository and contact Synthesis regarding their contributor agreement.