... until the collector arrives ...

This "blog" is really just a scratchpad of mine. There is not much of general interest here. Most of the content is scribbled down "live" as I discover things I want to remember. I rarely go back to correct mistakes in older entries. You have been warned :)


Ecmascript RegExp

Here are a couple of fine points about Ecmascript regular expressions worth noting.

The '.' atom does not match the four line terminator characters LF, CR, LS, and PS.  This is true irrespective of the presence of the multiline flag in the expression.

You indicate that a quantifier should be non-greedy by suffixing the pattern with '?'.  For example, '.*?', '.+?', '.??' etc.

These points are true for most other regular expression implementations as well.



If you invoke a Windows script directly from a command prompt, it will be launched using WScript and its exit status (set using WScript.Quit) will be gobbled.  If you launch it using CScript, the exit status will be returned properly.  Interestingly, if you invoke a script file from within a batch file, CScript is used automatically, despite the default file association to WScript.


BAT Files vs. Variable Expansion

You might think that the following BAT script would print out the word right:

set answer=wrong
if "a" == "a" (
    set answer=right
    echo %answer%

... but it actually prints out wrong.  The reason is that the BAT processor expands the variable answer when it is read, not when it is executed.  To work around this problem, you must use an extension, thus:

setlocal enabledelayedexpansion
set answer=wrong
if "a" == "a" (
    set answer=right
    echo !answer!

Note the use of exclamation marks instead of percent signs to delimit the delayed expansion of the variable answer.



TCPMon has been moved out of the Axis project and into a subproject of its own in the Apache Web Services commons.

Blank JMeter Response Data

I was testing a web application using JMeter, and the Response data tab of an HTTP Request sampler was always empty.  It turns out that JMeter does this if the content type of the response is not in a blessed list of "text" responses.  In my case, the content type was application/xml which is not on the list by default.  To fix it, edit the jmeter.properties file to include a line like this:


This directive specifies additional types to supplement the built-in types (text/*, etc). The shipped jmeter.properties file comes with a template line for this property.



In SQL Server 2005, the function OBJECT_ID takes a second argument which specifies the type of the object (e.g. 'U' for user table).  It turns out that SQL Server 2000 will also accept this parameter (at least, SP3 will).  This is not documented behaviour and does not work if you attempt to compile a procedure that calls OBJECT_ID with the second parameter (you get the message 'Invalid parameter 2 specified for object_id).


Line Breaks in Mathematica

In Mathematica, running on Windows, I wanted to generate a text file using UNIX newline conventions.  Unfortunately, the Export / Text function does not provide an option governing line terminators -- you always get the convention of the host platform.  As a workaround, if you export using String format, then the file will use Mathematica's convention, which is UNIX-style.


MS SQL Server Executable Code

The following query will list all user-defined executable objects (procedures, functions, views):

select name
from sysobjects
where objectproperty(id, 'IsExecuted') = 1
and objectproperty(id, 'IsMSShipped') = 0


Dynamic Test Suites in JUnit 4

Here is how you create a dynamic test suite using the JUnit 4 (instead of the old JUnit 3 suite mechanism):

public class MyRunner extends Runner {

    private static final Description TEST1 = Description.createSuiteDescription("test1");
    private static final Description TEST2 = Description.createSuiteDescription("test2");

    public MyRunner(Class<?> testClass) {
        // required by JUnit

    public Description getDescription() {
        Description description = Description.createSuiteDescription(MyRunner.class);
        return description;

    public void run(RunNotifier notifier) {
        notifier.fireTestFailure(new Failure(TEST2, new RuntimeException()));




BeanSetter.java contains a class that converts a table of strings into a list of Java beans.

DOM getElementById vs HTML Parsing in Java

If you parse an HTML document using JTidy, the DOM instance that it returns only supports DOM level 1.  The DOM level 2 method  getElementById is "supported", but it unconditionally returns null.

Using htmlcleaner 1.55 and Java 1.5, a native Java DOM object is returned.  It supports getElementById, but it will not recognize the id elements defined by HTML.  Thus, it also unconditionally returns null.  Java 1.5 uses Xerces internally, so it is probably possible to use some of the Xerces HTML parsing capabilities, but this would be an unsupported solution.


Windows vs USB Drives

Under Windows, you might plug in a USB pen drive, but it does not appear.  No error message is given, and the device is listed in the Safely Remove Hardware list, but you cannot find it in Explorer.

A possible cause for this behaviour is that the drive is trying to use a drive letter that is already assigned to something else.  You can check this by right-clicking on My Computer, selecting Manage, then Disk Management.  Look for the drive.  If it is there, right-click on it and use the Change Drive Letters and Paths option to assign a different drive letter.

SQL Server: Offline vs DROP DATABASE

If you issue the command DROP DATABASE for a SQL Server database that is offline, the associated MDF and LDF files will not be deleted.

In a related note, if you take a database offline using

sp_dboption 'mydb','offline','true'

you may get this error message when you attempt to bring it online:

Device activation error.  The physical file name 'c:\blah\mydb.ldf' may be incorrect.

Use the following command instead:

alter database mydb set offline with rollback immediate

The former appears does not appear to offline the database files cleanly.


Copying a SQL Server database

The following T-SQL sequence will make a copy of a database using the BACKUP and RESTORE commands:

backup database mydb to disk = 'c:\myfile.dat'

restore database mydbcopy from disk = 'c:\myfile.dat'
with move 'mydb_dat' to 'c:\my_dbcopy.mdf'
, move 'mydb_log' to 'c:\my_dbcopy.ldf'

The magic names referenced in the MOVE...TO clauses are logical filenames.  You can determine these by running sp_helpfiles in the source database (or querying the sysfiles table).  The following commands will do the same thing using file copies instead:

exec sp_detach_db @dbname='mydb', @skipchecks='true'

exec master..xp_cmdshell 'copy c:\mydb.mdf c:\mydbcopy.mdf'
exec master..xp_cmdshell 'copy c:\mydb_log.ldf c:\mydbcopy_log.ldf'

exec sp_attach_db @dbname='mydb', @filename1='c:\mydb.mdf', @filename2='c:\mydb_log.ldf'

exec sp_attach_db @dbname='mydbcopy', @filename1='c:\mydbcopy.mdf', @filename2='c:\mydbcopy_log.ldf'

This is slightly messier, but runs a bit faster (say, 40%).



I created a simple HTA for executing SQL queries against a SQL Server instance.


Oracle and SQL Server Table Functions

For reference, here are examples of table-valued functions in both Oracle and SQL Server


create or replace type range_row_type as object(n number);

create or replace type range_table_type as table of range_row_type;

create or replace function number_range(startNumber in number, endNumber in number)
return range_table_type
  result_row range_row_type := range_row_type(0);
  for i in startNumber .. endNumber loop
    result_row.n := i;
    pipe row(result_row);
  end loop;

select * from table(number_range(10,20));

SQL Server:

create function number_range (@start int, @end int)
returns @sequence table (n int)
as begin
  declare @i int
  set @i = @start
  while @i <= @end begin
    insert into @sequence(n) values(@i)
    set @i = @i + 1

select * from number_range(10,20)


Searching SQL Server Procedure Definitions

The SQL Server 2000 implementation of INFORMATION_SCHEMA.ROUTINES only contains the first 4000 characters of the procedure definition (apparently this is fixed in SQL Server 2005).  This makes it difficult to search for strings in the procedure text.  If you search in sysobjects and syscomments instead, there will be multiple rows per procedure called, one for every 4000 characters of the procedure definition.  At least this makes the text accessible.  Here is an SQL statement that will search the definitions of functions, procedures, views, and triggers for a string (provided the string is no more than 50 characters long):

select so.name
from sysobjects so
inner join syscomments as sc on (sc.id=so.id)
left join syscomments as overlap on (overlap.id=so.id and overlap.colid=sc.colid+1)
where so.type in ('FN','IF','P','TF','TR','V')
and sc.text+coalesce(substring(overlap.text,1,50),'') like '%mystring%'

Note the neat trick of self-joining syscomments to the next row in sequence to give 50 characters worth of overlap in case the search string spans rows (adjust the 50 to taste).

This is way easier than some of the expert advice about this topic on the 'Net.


Eclipse/OSGI vs. JUnit

It is standard practice to place unit test source code in a directory tree distinct from the "real" source.  In those rare cases when you want to test package-private features, the standard trick is to put the unit test class in the same package as the class under test (albeit in a different directory).  This scheme works well in "raw" Java.

The trick also works in Eclipse using a JUnit Test launch configuration, even if the unit tests are in a separate plug-in (another standard practice).  However, if you use a JUnit Plug-in Test launch configuration instead, you will get an IllegalAccessException.  Apparently, OSGI prevents one plug-in from contributing classes to a package defined in another plug-in.  Thus, under OSGI, packages are effectively sealed as seen from other plug-ins.



JExcelAPI is a pure Java library for reading and writing Excel files.


BIRT Support Site

BIRT Exchange is a new site offering BIRT-related support.

Windows Font-Size Scales Images

If you change your Windows display driver to use large fonts, images may also be scaled -- at least in IE.  I found a situation where a graphic in an embedded IE instance looked awful.  It was larger than expected, and had been scaled with the resulting jaggies.  It turned out to be due to selecting large fonts.  Note that not all applications seemed to be affected this way, but IE7 definitely was.  The display driver in question came from ATI.

BIRT Aggregate Functions

BIRT provides a number of aggregation functions, such as Total.sum() that can be called from within Javascript expressions.  But be aware that these so-called "functions" are actually macros, and are expanded by BIRT prior to execution by the Javascript engine.  The means that they can only appear at the top level of a BIRT data field expression and similar contexts.  They cannot be used in normal Javascript functions or any other non-top-level context.

BIRT Report Data Sources

Here is a code snippet that will replace the design-time connection parameters in all of the ODA/JDBC data sources that use a particular driver:

private void fixupDesignTimeJdbcDataSources(ReportDesignHandle reportDesignHandle) {
    String jdbcDriver = "my.driver.Class";
    String jdbcUrl = "jdbc:mynewurl";
    String jdbcUser = "myuserid";
    String jdbcPassword = "mypassword";
    for (Iterator<?> d = reportDesignHandle.getDataSources().iterator(); d.hasNext();) {
        Object dataSource = d.next();
        if (dataSource instanceof OdaDataSourceHandle) {
            OdaDataSourceHandle odaDataSource = (OdaDataSourceHandle) dataSource;
            if (ODA_JDBC_EXTENSION_ID.equals(odaDataSource.getExtensionID())
                    && jdbcDriver.equals(odaDataSource.getProperty(ODA_DRIVER_CLASS))
                    && null != odaDataSource.getProperty(ODA_URL)) {
                try {
                    odaDataSource.setProperty(ODA_URL, jdbcUrl);
                    odaDataSource.setProperty(ODA_USER, jdbcUser);
                    odaDataSource.setProperty(ODA_PASSWORD, jdbcPassword);
                } catch (SemanticException ex) {
                    throw new RuntimeException(ex);

private static final String ODA_JDBC_EXTENSION_ID = "org.eclipse.birt.report.data.oda.jdbc"; //$NON-NLS-1$
private static final String ODA_DRIVER_CLASS = "odaDriverClass"; //$NON-NLS-1$
private static final String ODA_URL = "odaURL"; //$NON-NLS-1$
private static final String ODA_USER = "odaUser"; //$NON-NLS-1$
private static final String ODA_PASSWORD = "odaPassword"; //$NON-NLS-1$

BIRT Life-cycle Summary

The Eclipse site has a nice article summarizing the BIRT Report Scripting life-cycle.

Dynamic Chart Labelling in BIRT

Here is an example script that can be attached to a BIRT chart object in order to change the labels on the chart dynamically:


function beforeDrawAxisTitle(axis, label, icsc)
    var title = label.caption.value;
    if ("" == title) {
        if (axis.orientation === Orientation.get(Orientation.HORIZONTAL)) {
            label.caption.value = ReportUtilities.formatLongDate(getParameterValue("fromDate"))
                + " to " + ReportUtilities.formatLongDate(getParameterValue("toDate"));
        } else {
            label.caption.value = getParameterValue("currencyDesc");

function beforeRendering(gcs, icsc)
    var chart = gcs.chartModel;
    chart.title.label.caption.value = getParameterValue("reportingPeriodDesc") + " Capital Spending";


CruiseControl vs Subversion

When CruiseControl attempts to determine the modification set using Subversion, it may quietly fail.  'Quietly' in the sense that the build will be reported as a success.  However, if you check the CruiseControl log, a nasty Subversion exception will be reported even though the build did not fail:

2007-09-14 23:40:28,319 [Thread-5128] ERROR SVN - Error executing svn log command svn log --non-interactive --xml -v -r {2007-09-14T21:32:04Z}:{2007-09-15T05:40:28Z} --username autobuild svn://mm:3691/trunk
org.jdom.input.JDOMParseException: Error on line 3: XML document structures must start and end within the same entity.
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:468)

The CC log does not show the actual error from Subversion. The problem appears to be that SVN issues an error or warning message prior to or in lieu of its XML output. CC, however, seems to parse out partial results and returns a bogus count of modifications. As a result, the build will continue using source code whose exact revision level is undetermined.


SQL Server Day Vector

Here is SQL server code to generate a day vector:

create table day_vector (
  date_index bigint primary key,
  full_date datetime unique,
  year_part int,
  month_part int,
  day_part int

declare @first_date datetime, @last_date datetime
set @first_date = '2000-01-01'
set @last_date = '2050-12-31'

declare @date datetime
set @date = @first_date

while @date <= @last_date begin
  declare @date_index int
  set @date_index = datediff(day, @first_date, @date)
  insert into day_vector
           (date_index,  full_date, year_part,             month_part,             day_part)
    values (@date_index, @date,     datepart(year, @date), datepart(month, @date), datepart(day, @date));
  set @date = dateadd(day, 1, @date)




WinDirStat is a nice utility for inspecting and cleaning up disk space usage.  Similar to TreeSize, but with more visualization features.


Granting EXECUTE to SQL Server User-defined Functions

In SQL Server, you can grant EXECUTE or REFERENCES to a scalar-valued user-defined function.  But if the function is table-valued, then only table permissions can be granted to it, not EXECUTE.

This consideration arose where I was programmatically granting EXECUTE to every user-defined function created by a script.  You can identify whether a function is scalar or table-valued with this query:

select objectproperty(id, N'IsTableFunction') from sysobjects where name=?


Installing WebLogic 8.1 as a Service

When you use the WebLogic 8.1 command installService.cmd, make sure that you first run the corresponding setDomainEnv.cmd first.  If you do not, then the service will be installed without the proper classpath settings (and possibly other settings as well).

Also, there is a command-line argument -execdir:"%USERDOMAIN_HOME%". that is used by the command beasvc within installService.cmdbeasvc seems to do some non-standard processing of the command line.  In particular, if %USERDOMAIN_HOME% ends with a backslash, then the closing quote does not actually close off the string.  It is as if the backslash is escaping the quote.  As a result, the service will be installed with an invalid working directory, one that includes all text on the command line up until the next closing quote.


IEEE 754 Double to Rational Conversion

Here is an algorithm to convert an IEEE 754 double-precision floating point number to a rational (in Java, using the JScience number types):

    private static final Rational doubleToRational(double value) {
        long bits = Double.doubleToLongBits(value);
        int sign = ((bits & 0x8000000000000000L) == 0 ? 1 : -1);
        int fractionBits = 52;
        int bias = 1023;
        int exponent = (int) (((bits & 0x7ff0000000000000L) >> fractionBits) - bias);
        long significand = bits & 0x000fffffffffffffL;
        if (exponent == (bias + 1)) {
            throw new ArithmeticException("cannot convert NaN or infinity to a rational number: "
                    + value);
        if (0 != exponent) {
            significand += 0x0010000000000000L;
        exponent -= fractionBits;
        LargeInteger numerator;
        LargeInteger denominator;
        if (exponent >= 0) {
            numerator = LargeInteger.valueOf(2).pow(exponent).times(sign * significand);
            denominator = LargeInteger.ONE;
        } else {
            numerator = LargeInteger.valueOf(sign * significand);
            denominator = LargeInteger.valueOf(2).pow(-exponent);
        return Rational.valueOf(numerator, denominator);

Java BigDecimal

Here is another reason why Java's BigDecimal is not very useful.  Try this:

BigDecimal.ONE.divide(new BigDecimal("3"))

You'll get this exception:

java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.

Java sorely needs a rational type.


BEA WebLogic 8.1 vs license.bea

If you manually move your BEA installation directory somewhere else, the WebLogic server will fail to start with a license exception.  Further digging will reveal that it is looking for the license file, license.bea, in the old directory. 

WebLogic 8.1 seems to use a hard-coded pathname to the license file.  I could not find where it was hard-coded -- it was not in any of the generated start-up scripts.  Their installer must put it into a JAR somewhere, or maybe even compile up a class file that contains it.  There is, however, a work-around.  Add the following system property to the command that starts the WLS server:


Of course, to have a hope of this working, you must edit the various BEA scripts to point to the new directory as well (or, a novel thought, to use relative pathnames).


Windows BAT vs. Spaces in Filenames

The BAT construct FOR %%I in (%~dp0\somefile.txt) will fail if the current directory has a space in the fully qualified path.  It will interpret the path as a list of space-separated paths instead.  The usual solution, to put the path in quotes, will not work in this context since FOR treats a quoted string as a literal line to be processed instead of as a single filename.

The workaround is to use %~dps0 instead of %~dp0.  Thus, short 8.3 filenames will be used instead, eliminating the problem.


Splitting a Cross-product in Mathematica

In Mathematica, the following expression will split a relational cross-product back into its component parts (assuming that common keys are sorted together):

SplitCrossProduct[keys_, list_] :=
    #[[1, keys]],
    #[[Complement[Range[1, Length[list]], keys]]] & /@ #
  }& /@ Split[list, SameQ[#1[[keys]], #2[[keys]]] &]


Graph Gear

Graph Gear is a Flash plug-in that renders interactive graphs (of the node and arc variety).



The Joda Time project has been proposed as the basis for the JSR-310 Date and Time API (also see the JCP page).


JAMWiki is an implementation of MediaWiki in Java.


Controlling Java Visibility in Eclipse

If you configure the Eclipse Java compiler to issue warnings for missing Javadoc comments on non-overridden public members, it provides feedback that encourages one to minimize visibility declarations (e.g. favour private over package over protected over public visibility).  It is easier to downgrade a member from public to package visibility then to write an inane comment.


Greenshot is a nice little open-source, .NET utility enhances the usual Windows PRINT SCREEN capability so that you can take screenshots of screen regions.  You can also perform simple markup of the captured images.

Hibernate 3.3: Lazy Collections vs. Multiple Sessions

Using Hibernate, there is a problem with lazy-loaded collections in detached objects.  Consider this sequence of events.  In one session, load an object with a lazy-loaded collection property.  Close the session and open a new session.  Associate the object with the new session.  Access an element of the collection (e.g. myCollection.first()).  The following exception occurs:

org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: MyEntity.myCollection, no session or session was closed
	at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:358)
	at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:350)
	at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:343)
	at org.hibernate.collection.AbstractPersistentCollection.read(AbstractPersistentCollection.java:86)
	at org.hibernate.collection.PersistentSortedSet.first(PersistentSortedSet.java:93)

A quick trip in the debugger shows that the session is closed.  Apparently, attaching an entity to a session does not attach its lazy collection properties.  So, how does one do that?  Well, you could try Hibernate.initialize(myCollection).  But then you would get:

org.hibernate.HibernateException: disconnected session
	at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:452)
	at org.hibernate.Hibernate.initialize(Hibernate.java:309)

Perhaps mySession.lock(myCollection, LockMode.NONE)?  Nope, this gives:

org.hibernate.MappingException: Unknown entity: org.hibernate.collection.PersistentSortedSet
	at org.hibernate.impl.SessionFactoryImpl.getEntityPersister(SessionFactoryImpl.java:550)
	at org.hibernate.impl.SessionImpl.getEntityPersister(SessionImpl.java:1338)
	at org.hibernate.event.def.DefaultLockEventListener.onLock(DefaultLockEventListener.java:50)
	at org.hibernate.impl.SessionImpl.fireLock(SessionImpl.java:584)
	at org.hibernate.impl.SessionImpl.lock(SessionImpl.java:576)

Okay, surely this will work:

PersistentCollection collection = (PersistentCollection) myCollection;
collection.setCurrentSession((SessionImplementor) mySession);


org.hibernate.HibernateException: collection was evicted
	at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:38)
	at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1716)
	at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:454)
	at org.hibernate.Hibernate.initialize(Hibernate.java:309)

onInitializeCollection is calling PersistenceContext.getCollectionEntry(collection) and coming up empty -- the new session has no clue about the collection.

I'm stumped.  I can't figure out how to bring an uninitialized collection into a new session.  It looks like you must initialize a collection in the session that first accesses it.



TrueCrypt provides virtual encrypted disk functionality for Windows.  It can host a virtual filesystem within a file or across an entire device (including a flash drive).

Mathematica Bar Charts

On Mathematica bar charts, you can add tooltips showing the values for each bar thus:

BarChart[data] /.
  Rectangle[a_, {b_, c_}] :> Tooltip[Rectangle[a, {b, c}], c]


StAX support is growing, especially since the reference implementation found its way into Java 6.  However, the general buzz is that the reference implementation is not good enough for production use.  Here are some alternatives:


Pending Events in AWT and SWT

AWT does not provide a function to process pending UI events.  However, you can get the same effect by calling EventQueue.invokeLater(Runnable) which will not return until after all pending events are processed and the supplied Runnable finishes.  SwingUtilities also provides invokeLater.

SWT provides Display.readAndDispatch() to process all pending events.  This is a good thing, because trying the "invoke later" trick using Display.syncExec(Runnable) won't work.  SWT makes no guarantees about event processing in syncExec -- it simply ensures that the Runnable will be executed in the UI thread at "the next reasonable time".


IE vs. Invalid Colspan

From time-to-time I have seen IE render tables on web pages without drawing the border on the right-most column of cells.  Today I found out at least one cause for that behaviour.  It happens if you have a table cell with a colspan that is too large, but only in strict mode and when using CSS.


Mathematica Debugger

The new debugger in Mathematica 6.0 looks broken to the untrained eye (like mine).  If you load up all of your modules and then turn on debugging, you might be frustrated by the fact that none of your breakpoints work (even though they show visually and are listed in the breakpoint list).  Or by the fact that you cannot step into code.  It took me a while to figure out that it all works if you turn on debugging before you load your modules.

It would be nice if Mathematica gave an indication that a breakpoint was being ignored.  Or if the debugger documentation mentioned this gotcha.  Or if the debugger had any documentation at all :-)

Mathematica Pure Function Scope Problem

Consider this snippet:

f[x_] := g[Function[a, x]];
g[fn_] := Module[{h}, h[a_] := fn[a]; h[0]];

Executing this sequence generates errors messages:

Function::flpar: Parameter specification {0} in Function[{0},999] should be a symbol or a list of symbols. (x3)
$RecursionLimit::reclim: Recursion depth of 256 exceeded. (x4)


However, if you change the name of the function variable in the first line from a to z, then all works as expected:

f[x_] := g[Function[z, x]];

=> 999

Another workaround is to compile the function:

f[x_] := g[Compile[a, x]];

=> 999

The problem appears to be caused by a name conflict between the pure function's argument a and the helper function h's argument a.  This is a really nasty problem because the definitions are completely separate -- one would have to perform a global code analysis to turn up such problems.

Windows Media Encoder

Windows Media Encoder has a feature whereby you can publish streaming video of your screen via HTTP or MMS.  You can use Screen Capture as a source and Pull from encoder as an output.  You may have to fiddle with the compression options to get the right balance of quality and bandwidth.  Use the Edit button for a custom compression set-up under the Compression tab -- especially to set the video size to be the same as the input.    The frame rate can be really low -- 3fps doesn't look bad.  Note that you will probably have to reduce screen resolution to 800x600, 16-bit colour to avoid a cryptic "not supported" error.

There is a nice "how to" article on the Microsoft site, Getting Started with Screen Capture Using Windows Media Encoder.



Quantrix is an Excel-like modelling tool that consolidates formula handling (so that you do not have to read the formula in every cell to see what's going on).  The pivot table functionality is a bit slicker than Excel as well.



EclEmma is an Eclipse plug-in for the EMMA Java code coverage tool.


SQL Server 2000 vs Executing Text Values as SQL

In SQL Server 2000, it is possible to execute a long SQL statement that is stored in a text column in a table.  It is just ugly.  Take a look:

declare @s0 varchar(8000)
declare @s1 varchar(8000)
...boring lines omitted...
declare @s18 varchar(8000)
declare @s19 varchar(8000)
select @s0=substring(sqlText, 1, 8000)
, @s1=substring(sqlText, 1+8000, 8000)
...more boring lines omitted...
, @s18=substring(sqlText, 1+18*8000, 8000)
, @s19=substring(sqlText, 1+19*8000, 8000)
from tableWithSqlInIt
exec (@s0+@s1+@s2+@s3+@s4+@s5+@s6+@s7+@s8+@s9+@s10+@s11+@s12+@s13+@s14+@s15+@s16+@s17+@s18+@s19)

The exhibited statement only works for text columns whose values are no longer than 160,000 characters (20 x 8000).  Adjust to taste.  Also, if you are using NTEXT columns, then you must use NVARCHAR(4000) and segment the value into blocks of 4000 to avoid overrunning SQL Server's 8000 byte limit.


Hibernate Session.isDirty()

Hibernate's Session class has the method isDirty() which tells you whether there are any changes that still need to be synchronized with the database, i.e. that there are still pending operations.  Perhaps it is obvious to some, but I was caught thinking that it meant that the current Hibernate transaction contains uncommitted changes.  It doesn't.  If you want to know that, you must keep track of it yourself (perhaps by registering some listeners within Hibernate).


Sharing Scripts in BIRT

You can share Javascript files across BIRT reports by manually editing the report layout XML.  Add lines like:

 <list-property name="includeScripts">

Relative pathnames are supported. The visual designer does not (yet) support script inclusion directly.


Mercurial and SVK Version Control

Mercurial is a new version control system that specializes in distributed development.  Its distinguishing feature is that there is no central repository -- all local repositories are peers (like Gnu arch).  Synchronization occurs through change sets.  It is growing in prominence because the Mozilla 2 team has chosen to use it in favour of CVS.

When reading up on Mercurial, I came across SVK, which is an extension of SVN to support distributed development.


Java 1.5 / Xalan XSLT Bug

Xalan XSLTC 2.6.0 (and by extension, Java 1.5 XSLT transformation) has a bug where it complains about a syntax error for a rule such as <xsl:template match="and"/>.  The issue is that and is a reserved word.  Only Xalan appears to exhibit this bug.  All of the other transformers that I tried worked (including IE, FF, Saxon, and Xalan 2.7.0).  The XSLT specification is silent on the matter, but the grammar does not appear to prohibit the case.

The workaround is to use a rule of the form <xsl:template match="*[local-name()='and']"/>.  Ugly, but it works.


Enabling SVN Log Editing

You can enable Subversion log editing by installing a pre-revprop-change hook.  Here is a minimal Windows batch file that will do the trick (converted from the template UNIX shell script that comes with SVN):

@echo off

set REPOS=%1
set REV=%2
set USER=%3
set ACTION=%5

if "%ACTION%" == "M" if "%PROPNAME%" == "svn:log" exit 0

echo Changing revision properties other than svn:log is prohibited >&2
exit 1


Empty Node Set in XPath 1.0

'/..' is an idiomatic way to express an empty node set in XPath 1.0.  In XPath 2.0, one can use the empty sequence '()'.

Event-Handling in FF and IE

The following snippet can be used in either Firefox or Internet Explorer to recover the target document node from a click event:

window.onload = function() {
    document.body.onclick = function(suppliedEvent) {
        var target;
        if (undefined != suppliedEvent) {
            target = suppliedEvent.target; // Firefox
        } else if (undefined != event) {
            target = event.srcElement; // IE
        } else {
            return; // some other browser?


Little Languages

Little languages are in the news a lot these days (under the rubric DSL or Domain Specific Language).  I came across A Pattern Language for Language Implementation.


Sun has blessed the F3 project with the new name Java FX.  It is being represented as a competitor to Flash and Silverlight.  The implication is that FX will find its way into the JRE.


Grepping for static fields in Java

Here is a regular expression (Java-syntax) that will match all static non-final fields in Java source code (to a reasonable approximation):


In summary, if finds all lines that contain the word static, but not followed by  final, class, enum, or an open parenthesis (signifying a method) prior to an equals sign.  This pattern will return false positives such as when static appears after an equals sign or final precedes static.  It will also miss cases that are split over multiple lines.  Both circumstances are unusual to the point of rarity and a more elaborate pattern to find such cases will run considerably slower.

For some reason this expression works erratically within Eclipse.  Eclipse seems to occasionally search across line boundaries -- but only sometimes?!  To correct his behaviour, use this pattern instead (changes in bold):



Subversion: cannnot replace a directory from within

I tried to use TortoiseSVN to merge a particular revision of a particular file from one branch into another.  I received the message "cannot replace a directory from within".  I tried the same thing using the SVN command line tools, same result.  The problem was that I was using a command like this:

svn merge -r3023:3055 svn://myrepository/trunk/blah/blah/Blah.java c:\myworkingcopy

The solution was to specify the target filename directly:

svn merge -r3023:3055 svn://myrepository/trunk/blah/blah/Blah.java c:\myworkingcopy\blah\blah\Blah.java

You can do this from within TortoiseSVN by invoking the 'merge' command by right-clicking directly on the file in question instead of at the root of the working copy.


SQL Server SP2 vs TCP/IP

There is a problem with SQL Server SP2 that prevents its operation using TCP/IP.  Port 1433 (or the configured port) is not open.  If you check the application event log, you will find this nice message:

You are running a version of Microsoft SQL Server 2000 or Microsoft SQL Server 2000 Desktop Engine (also called MSDE) that has known security vulnerabilities when used in conjunction with this version of Windows. To reduce your computer's vulnerability to certain virus attacks, the TCP/IP and UDP network ports of Microsoft SQL Server 2000, MSDE, or both have been disabled. To enable these ports, you must install a patch, or the most recent service pack for Microsoft SQL Server 2000 or MSDE from http://www.microsoft.com/sql/downloads/default.asp

More information from this issue can be found in KB841375.



The Joda project seeks to improve on core Java functionality.  Of particular interest is the Joda Time package, which is meant to replace the Java's messy date and time handling.

Fillers in SWT GridLayouts

SWT GridLayouts require every cell to have content.  If you want an empty cell, add a blank label.  If you try to add a blank composite, the cell will likely be larger than what you want since composites seem to have a somewhat large minimum size (at least in a grid layout).


Generating Random Numbers in T-SQL

The SQL Server T-SQL function RAND() generates random numbers, but it will only generate exactly one such number in a given query (i.e. multiple 'calls' to the function all return the same number).  Another way is to exploit the NEWID() function which returns a UUID (GUID).  At time of writing, the function returns Version 4 UUIDs, which means that most of the bits are random (note the variant or version bits, see the UUID quick summary, below).

So to get a four byte random number, you can use:

select substring(cast(newid() as binary(16)),1,4)+0

Be careful if you try to extract more bits since it is tricky figuring out which bits are which, viz:

  2BDB9E2A-A20D-43F9-9117-2ABCEBD6434E cast to binary(16) is
0x2A9EDB2B 0DA2 F943 9117 2ABCEBD6434E

Note that the leading eight hex octets are switched to low-endian component-wise -- but the other eight octets are not.  The less-than-random hex digits are bolded.

UUID Quick Summary

The layout of a UUID is as follows:



TTTL = 4 hex octets, "time low"

TM = 2 hex octets, "time mid"

TH = 2 hex octets, "time high and version", "version" in 4 most significant bits (0001 = time-based, 0010 = DCE, 0011 = name-based/MD5, 0100 = random, 0101 = name-based/SHA-1

R = 1 hex octet, "clock seq high and reserved", "variant" in 1-3 most significant bits (0 = NCS [1-7], 10 = RFC 4122 [8-B], 110 = Microsoft [C-D], 111 = reserved [E-F])

C = 1 hex octet, "close seq low"

NNNNNN = 6 hex octets, "node"

On the wire, the bytes of a UUID are transmitted in network byte order (most significant byte first)


SQL Constraint Violations

Here is a snippet that will identify whether a given Java exception is caused by an SQL constraint violation:

private static boolean isConstraintViolationException(Exception exception) {
    for (Throwable cause = exception; null != cause; cause = cause.getCause()) {
        if (cause instanceof SQLException) {
            String sqlState = ((SQLException)cause).getSQLState();
            if (null != sqlState && sqlState.startsWith("23")) {
                return true;
    return false;

The black magic is bolded.  All of the SQLSTATE values in the 23xxx block represent some form of constraint violation.  Many databases simply return 23000 which indicates a non-specific type of violation.


Java Class Initialization

I came across a interesting gotcha involving Java class initialization.  Consider the following:

abstract class Base {
    Base() {
    protected abstract void inner();

public class Derived extends Base {
    int x = 0;  // GOTCHA!
    int y;

    Derived() {
        System.out.format("x = %s, y = %s\n", x, y);

    protected void inner() {
        x = 666;
        y = 777;

    public static void main(String[] args) {
        new Derived();

The program prints out "x = 0, y = 777".  You might have expected x to be 666.  The culprit is the field initializer labelled GOTCHA! which is executed after the superclass' constructor.  Java takes great pains to prevent you from writing code like this (e.g. you cannot reference this when calling the super() constructor).  And this example shows why.  This is all made clear in the Java Language Specification, but here is a practical instance that actually occurred in the field.


Starting Oracle on Windows

Under Windows, to configure an Oracle instance to start automatically when its service starts, make sure that the following registry key is set:


where ID is the identification number of the Oracle home (typically '0') and INSTANCE is the name of the instance.  The corresponding service will be named 'OracleServiceINSTANCE'.

Also note that it is generally a bad idea to stop and Oracle instance by simply stopping the service.


Growing Java Arrays

Here is a Java convenience method to append an element to an array (of any kind):

private static <T> T[] appendToArray(T[] array, T element) {
    T[] newArray = (T[])Array.newInstance(array.getClass().getComponentType(), array.length + 1);
    System.arraycopy(array, 0, newArray, 0, array.length);
    newArray[array.length] = element;
    return newArray;

Hacking SVN Logs

I wrote a little Windows batch script, svnloghack,  that makes it easier to change the revision logs in a Subversion repository.  It takes a path to a subversion repository and a list of revision numbers as command line arguments.  It creates a file named zsvnhackrev.txt for each revision (where rev is the revision number).  Each file contains the corresponding log message.  It also creates a batch script named zsvnhack.bat that writes back all of the log messages to their corresponding revisions in the repository (presumably after you have edited them).

Also, the batch script pre-revprop-change.bat is a minimalist hook script for Subversion that will permit log changes (and other revision property changes) to take place.  You have to put it in the hooks directory of the repository if there isn't such a hook already there.


JUnit Tear-down Assertions

In JUnit 4.1, if an assertion failure occurs in a tear-down method it will mask any failure that occurred in the test case itself.  There is no real work-around for this problem other than commenting out the assertion in the tear-down method.


Offline Web Reading

A decent set of options for WGET to suck down web pages for offline reading is:

wget -r -l 0 -k -p -L -np -nv http://somesite.com/index.html

Adjust the -l option to taste (number of levels to descend, inf for infinity).


Time and Date

The web site http://www.timeanddate.com/ is a useful resource for, well, time and date information.  In particular, it has the niggling details about when different jurisdictions change to daylight savings (a pertinent topic with the North American DST change this year).


The Eclipse AspectJ development tools (1.4.1) seem to be finicky when it comes to applying changes to aspects.  If you uncomment a declare warning, for example, the change will not be noticed and will not trigger a rebuild even after you save the file.  You must make a more substantial change elsewhere in the aspect file.  Also, the builder seems to ignore changes to all aspect files after you add AspectJ nature to a Java project .  I found myself having to do manual cleans to get my AspectJ changes to take.

The Compiler is Your Friend

Strong typing has a bad rap these days.  Someday (read: never), I will have to write an article about how it is not evil.  Especially since strong typing was a big part of what made me "downgrade" from Lisp to Java.

Yesterday, I spent a couple of hours sweeping through our "application in crisis".  Being a Java application that uses RCP and Hibernate, there was a lot of run-time type checking going on.  Unnecessarily, I might add.  By declaring the real types of objects (instead of just Object), the compiler dutifully found a bunch of bugs that would have taken a whole lot of testing, or customer deployment, to find.  Bugs like: confusion over whether an entity's key was scalar or composite; using the wrong return type for a query; diverging from the mandated equals/hash code strategy; or trying to persist non-persistent objects.  All found at the cost of a few minutes of adding appropriate declarations.


JDBC Statement Types

The JDBC API says that CallableStatement extends PreparedStatement that extends Statement.  This suggests that CallableStatement can do everything that PreparedStatement can do, and more (just as PreparedStatement has a superset of Statement functionality).  Well, O-O principles suggest this (viz "extends"), but the documentation leaves room for doubt.  It says that CallableStatement is intended for use to call stored procedures.  Of course it is, except that the wording suggests that that is all it should be used for.  So I wrote a quick program that used the ODBC bridge to see whether I could run a simple query through a CallableStatement.  It worked, and I thought my paranoia unjustified.  Until I tried the same thing with the jTDS driver.  It didn't work, complaining about a syntax error at the keyword SELECT.

Transactions in Hibernate

If you are borrowing the connection from Hibernate (a questionable practice, to be sure), then be aware that JDBC Connection transactions are much faster than Hibernate Session transactions.  In our application, Session.getTransaction().commit() took almost a second when stepping over it in the debugger -- and that is when the only work done was a query.


BIRT vs Script Extensions vs jTDS vs Eclipse Plug-ins

BIRT permits developers to create script extensions in Java.  These extensions can be called from a report's script to perform customized processing.  However, BIRT will not find such extensions unless you manage the classpath yourself manually.  The problem can be corrected by editing the MANIFEST.MF in the plug-in org.eclipse.birt.report.engine and adding the directive Eclipse-BuddyPolicy: dependent to the end.  This allows BIRT to find extensions in your plug-ins.

BIRT also allows you to add JDBC drivers to the report engine by dropping files into a directory in the plugin org.eclipse.birt.report.data.oda.jdbc.  This does not work, however, for drivers that require native libraries (such as ntlmauth.dll that supports the jTDS driver).  You can work around that problem by creating a plugin that contains your driver and its native library, and then adding a reference to that plugin in the manifest of BIRT's JDBC plugin.  It is not sufficient to copy the native library into BIRT's plugin if your application uses it itself because native libraries can be loaded from exactly one plugin.

This all seems nice and neat, but here is a problem.  A developer experienced a problem where the jTDS driver class could not be loaded by BIRT (even though it had been successfully loaded by the main application).  Checking the application's plug-in registry, the jTDS plug-in had been successfully resolved.  I had run into a similar problem when I had switched into a branch that lacked the jTDS plug-in and then switched back.  I found that I could fix the problem by touching the MANIFEST.MF in BIRT's org.eclipse.birt.report.data.oda.jdbc plug-in.  I chalked it up to some sort of stale plug-in cache in Eclipse.  But the other developer did no such branch switch -- Eclipse just stopped respecting the updated manifest directive.  The 'touch' trick fixed it, however.  Puzzling...

This entry was updated on 2007-02-13: the plug-in to patch with Eclipse-BuddyPolicy is org.eclipse.birt.report.engine, not org.eclipse.birt.report.viewer.  If you change only the latter, then only the BIRT viewer component will see your extensions.  If the change the former, both in-memory and viewer applications will see them.

Excel User-defined Worksheet Functions

I don't know why I keep forgetting this: if you want to use a VBA function in a worksheet formula, the function must be defined in a module (neither workbook nor worksheet functions are visible).


Building esi.execute

I tried to do a build of a freshly checked out copy of esi.execute.  There were a number of problems:

- I needed to delete junit-4.1.jar from C:\esidev\eclipse3.2.x\plugins\org.apache.ant_1.6.5\lib

- I had to add the TestNG plugin (org.testng.eclipse_4.7.0.0) to C:\esidev\external_plugins\eclipse\plugins

- I had to define the Eclipse classpath variable WEBLOGIC_8_1 as C:/esidev/bea/weblogic81

Eclipse xdoclet Error

In an Eclipse project, I had this pesky error message:

The preferences for the xdoclet runtime does not point to a valid installation.

I wasn't using xdoclet in the project.  It turns out that this can occur if some tool (like the WST plug-in) had added xdoclet nature to the project and subsequently removed it.  I found a reference to this problem in the Eclipse bug tracker.  The bug entry suggested adding and removing xdoclet nature to clear up any left over problem markers.  I did this, but then I found out that my project actually had xdoclet nature, along with WST and JEM natures.  I guess at some point the project must have been opened by the WST plug-in.  The funny thing was that the project did not take on the characteristic look of a WST project in the package explorer.  I hand-deleted the various natures from the .project file and then all was well.


SQL Server 2000 vs. Hibernate 3

Our Hibernate 3.1.3 application started acting up.  The symptoms were after a call to Query.list(), one of the queried tables was being locked.  This is strange, given that the operation is read-only, and that it only happened on one of the many such calls.  The problematic query was probably the largest query in the system.  We ran the application in three different environments, but it only failed in one.  The failed environment was running SQL Server Developer Edition 8.00.194 (RTM).  The two successful environments were running SQL Server Developer Edition 8.00.2039 (SP4).  Our hypothesis is that this accounts for the difference.  Early versions of MSDE had issues closing cursors -- perhaps we've been bitten by that.


Hibernate Logging

With all the preliminaries out of the way, now I can talk about appropriate logging levels in Hibernate.  Setting org.hibernate.SQL to at least DEBUG/FINE shows the SQL statements that are executed.  Setting org.hibernate.type to at lest DEBUG/FINEST shows the values bound to variables in the SQL statements -- but be warned that this is very verbose.

The Hibernate manual details all of the logging properties.

JDK Logging

If I had a dollar for every time I've looked this up...

To configure JDK logging, you can either edit the file JAVA_HOME/lib/logging.properties, or you can define the system property java.util.logging.config.file to point to another file.  The file looks like this:

handlers = java.util.logging.ConsoleHandler,java.util.logging.FileHandler

java.util.logging.ConsoleHandler.level = FINEST
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter

java.util.logging.FileHandler.pattern = %h/java%u.log
java.util.logging.FileHandler.limit = 50000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter

.level = INFO
org.hibernate.SQL.level = FINE
org.hibernate.type.level = FINEST

The level specification in bold is a common gotcha - it limits the level of the messages that are shown on the console.  The levels are: FINEST, FINER, FINE, CONFIG, INFO, WARNING, SEVERE (and OFF).  More details can found in the javadoc for java.util.logging.LogManager.

Eclipse Applications and JConsole

To enable an Eclipse application for use with JConsole, create an .INI file for your application (e.g. myapp.ini) that contains the following lines:


log4j Logging

I'd like more dollars for looking up this as well...

To configure log4j logging, the file log4j.properties must be present on the classpath.  It looks like this:

log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

log4j.rootLogger=debug, stdout, file


Again, the level in bold limits what is written to the appenders.  The levels are TRACE, DEBUG, INFO, WARN, ERROR, and FATAL.  More information can be found in the log4j javadocs.


Hibernate 3.1.3 Failed Saves

Hibernate 3.1.3 has the following buggy behaviour.  If a Session.saveOrUpdate() fails, the object being saved is still marked for saving.  Thus, the next time Hibernate tries to synchronize with the data store, the save will be retried and (probably) fail again.  The object should not be so marked until the application elects to explicitly try to save again.  This appears to be fixed in Hibernate 3.2.


Eclipse Java Compiler Bug

The Eclipse Java compiler issues an inappropriate warning in the following code:

    public int zot(Object obj) {
        if (obj instanceof String) return 1;
        if (obj == null) return 2; // inappropriate warning on this line
        return 3;

It complains that obj cannot possibly be null -- even though it can.  Experimentation suggests that something about the instanceof operator confuses the compiler.


Eclipse Java Compiler Non-Null Warnings

The Eclipse Java compiler does some clever reasoning about the possible null state of variables.  For example, the following code will compile without complaint:

Object result = getMyObject();

However, the compiler will generate a warning that result may be null in the following:

Object result = getMyObject();
System.out.println(null == result);
result.doSomething(); // warning: result may be null

The two look the same semantically, don't they?  It would appear that the compiler, as an expediency, presumes that any object returned from a call will be non-null, until it sees evidence that there is any other possibility.  In the second instance, there is such evidence because the programmer has expressed concern explicitly.  A clever trick, to be sure.

Hibernate vs. Object.equals()

You must take care when implementing Object.equals() for Java objects being persisted using Hibernate.  Well, this is no surprise because the Hibernate documentation says so.  However, here is a subtle gotcha that is not emphasized in the current Hibernate documentation.  I ran across some classes whose equals() method tested that the other object was of exactly the same class as the target object.  This is unusual, because most equals() methods settle for instanceof.  The ones that used strict equality were broken because Hibernate's decorated versions of classes might be used in place of the actual classes.  Consider, for example this stack trace:

AbstractPersistentObject$$FastClassByCGLIB$$1ce0f1a2.invoke(int, Object, Object[])
MethodProxy.invoke(Object, Object[])
CGLIBLazyInitializer.intercept(Object, Method, Object[], MethodProxy)

In this case, the initial call to equals() was passed a decorated object but you can see from the trace that it ultimately was passed to an undecorated version of the class.


IE7 vs. Automatic Windows Authentication

As of IE7, any URL whose host portion contains a dot is considered to be outside the intranet zone.  Thus, automatic Windows authentication is disabled (by default).  As a workaround, you can now add individual URLs to the list of intranet sites.

I tried to add a wildcard for the local domain to the list of sites (*.mm.local).  IE complained that the URL was already listed in another zone, and that I should remove it from that zone first.  True enough, it was in the trusted sites zone.  So I removed it and tried again.  I exited and relaunched IE.  I logged out and logged back in.  I rebooted.  The same message occurred after each of these measures. I tried a different wildcard, and it worked.

There appears to be a bug where once something has been in the trusted sites zone, you can't ever add it to another zone.  I connected to a web site in the local domain, entered the requested password, and IE showed the site as still belonging to trusted sites.  But the wildcard no longer appears in the trusted sites list.

Problem solved.  The registry key

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap

contains the IE zone mapping.  It didn't show my local domain, but the corresponding key under HKLM did.  I'm not sure how the HKLM key got there, and there was no evidence of the registry setting in the IE configuration dialogs.  Deleting the key made things work as expected.

Blog Archive