Skip navigation

Category Archives: Java

Ok, here’s a quick game! Let’s spot software releases which happened in the first days of December, 2008.

1. WordPress 2.7 –http://en.blog.wordpress.com/2008/12/05/new-dashboard-design/

2. Python 3.0 – http://www.python.org/download/releases/3.0/

3. JavaFX 1.0 – http://www.sun.com/aboutsun/pr/2008-12/sunflash.20081204.1.xml

Advertisements

With modern servers, a lot of people are migrating to 64-bit architectures. Apparently, there are performance considerations in regards to using a 64-bit JVM. If you are using/considering installing a 64-bit JVM, you might want to read up on compressed oops (ordinary object pointers) — don’t worry, we are only talking about JVM command line options which affect performance. Please visit the links below:

http://www.lowtek.ca/roo/2008/java-performance-in-64bit-land/
http://blog.juma.me.uk/2008/10/14/32-bit-or-64-bit-jvm-how-about-a-hybrid/
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=/com.ibm.java.doc.user.lnx.60/user/garbage_compressed_refs.html

a Postgresql Bulk Updater in Java

pgBee is a set of Java classes I wrote for automating bulk updates of Postgresql databases on Linux servers. It requires Java (doh!) and Ant (as a build/execute front-end), it is cronnable and performs very well, especially in multi-threaded mode, which takes full advantage of multi-core CPUs in modern servers. The source of inspiration for pgBee has been previously described.

This code is released under a GNU General Public License (GPL).

Ant sometimes refuses to run in the background, so the best way to make pgBee work as a cron job is probably to call a simple shell script from cron, like the one below:

#!/bin/bash
export JAVA_HOME=/usr
export ANT_HOME=/usr/local/ant
/usr/local/bin/ant -f /path/to/build.xml run </dev/null &

All configuration is done in the settings.xml file, but some options may be set through the command line, e.g.

ant -f /path/to/build.xml -Dlock=yes -Dthreads=8 -Dparallel=yes run

pgBee processes all files it finds in a particular (in) directory and moves them to either a done directory or a rejects directory, if there were SQL errors. You’ll need to create the right directory structure and configure pgBee settings before starting. The pgBee process catches SIGTERM, SIGHUP etc. signals and exits gracefully, ready to resume from where it stopped the next time it is run. So, it should be quite reliable, in the absence of hard resets and kill -9. Having said that, I am supplying no guarantees of fitness for any purpose of any kind ūüôā Please use at your own risk.

If you need to make sure a particular set of statements is processed in the same transaction, you only have to include all statements in the same line of an input file, separated by semi-colons. There’s no limit to how many SQL statements you may include in a single line. More information about input file format, usage and configuration may be found in the downloadable tarball

Over the last few years, I have very consciously shifted away from web interfaces into what most people call the back-end: systems, databases etc. I had grown tired of browser incompatibilities and unpredictability!¬†You may find trouble in server-side¬†land, too, but, in most cases, you’ve done something wrong and there is a perfectly reasonable, rational (and sometimes obscure) explanation waiting to be discovered.

So, I am completely out-of-touch with recent UI advances, as simple CSS + Javascript pages and extensive Perl-Tk and pygtk coding probably don’t count as UI advances for most people. And I tend to¬†code things the hard way: ViM

Now, Java is trying to re-enter the RIA arena with JavaFX and reclaim some rich UI tech real-estate from frameworks such as Adobe Flex and Microsoft Silverlight. And thanks to Peter Pilgrim and Skills Matter, I have just had a very interesting introduction to the history, concepts and potential of JavaFX.

Now, Chris Oliver, the guy who started F3, which has now developed into¬†JavaFX, sounds exactly like my kind of guy: Form-Follows-Function, list comprehensions, SQL influences, declarative stuff. Pre-compiled UI definitions running¬†on top of the JVM… Imagine, if you like, writing something like JavaScript, with syntactical goodies derived from Lisp and SQL, including UI element binds and definable triggers, which may execute any arbitrary Java code. Now, for me, this puts fun back into computing (and Java, which often is the C++ of our times). It’s not quite stable yet, but there should be a Preview release coming out end of this week from Sun Microsystems.

Now, I don’t think I am making this appealing to the majority of RIA developers. They usually think: timelines, SceneGraphs, rich media, GUIs, WYSIWYG, ActionScript etc. JavaFX, of course, has been developed to handle such concerns.

But to me, it seems like JavaFX might turn out to be the framework of choice for people who hate coding for UIs (but love coding in general), people who want them to be easy and fun, yet very powerful. I am, probably, one of the very few people who write Java in vi. Projects like Ant have made this possible.

If JavaFX matures in the current direction, it looks like I might be able to get away with designing rich UIs in vi, too. And before any of you start screaming: we don’t care about writing Java UIs using vi, may I simply point out: isn’t this the most convincing argument¬†for the promised power of¬†JavaFX? – they must be doing something right!

Yet another work-related post. I have been asked to write a better automatic database update system and against my natural tendencies toward Perl and Python I have opted to do it in Java. Now, previous attempts in Java had been abandoned because they were not performing very well, but I wanted to build something with potential for integration with the company’s infrastructure, so I rolled up my sleeves and decided to investigate.

A quick Google search produced some interesting discussions (please see the Interesting Links below). In summary, the official JDBC Postgresql driver does not support COPY operations and people complain that it’s slow for bulk updates, however, our update sql files are not very structured and, in fact, may contain any (as in different each time) valid SQL code. So, COPY is not what I’d use, anyway.

Some hope for reasonable performance appeared in the form of the driver’s batch mode. So, I wrote some Java classes which read multiple lines of sql statements from an sql text file into a String buffer of configurable size. When this size is reached, these sql statements are added to the reused Statement object with addBatch() and are executed in their own transaction (I have set auto-commit to off) through executeBatch().

Now, I have tried inserting one million rows into a table using a different buffer size each time, i.e. grouping sql statements in batches of one, ten, hundred and thousand statements per transaction. The results are quite promising, don’t you think? (low spec machine, btw)

  • batches of 1 –> 49m 55s
  • batches of 10 –> 15m 04s
  • batches of 100 –> 08m 21s
  • batches of 1000 –> 33m 12s

Interesting links (References):

multi-statetement JDBC updates in batch mode: http://archives.postgresql.org/pgsql-jdbc/2007-04/msg00076.php

making batch updates in JDBC applications: http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp?topic=/com.ibm.db2.udb.doc/ad/tjvbtupd.htm

no copy from postgres JDBC: http://archives.postgresql.org/pgsql-jdbc/2004-06/msg00027.php

copy for PostgreSQL 8.x JDBC Driver: http://kato.iki.fi/sw/db/postgresql/jdbc/copy/