Sep 25 2006

Building a better BeanProcessor

Tag: , warren @ 7:28 pm

At the moment I’m coding up a ‘BeanMapper’. Something to automatically transform a ResultSet into a JavaBean object. (Why would you want to do this? Write and maintain less code will probably do as a worthwhile cause.)

The DBUtils Apache commons project has a BeanProcessor that does a basic job. It is good for a simple mapping, however it does not meet the following challenges:

  1. Inheritance – a ResultSet may map to different Objects of the same parent over different rows
  2. Aggregates – a ResultSet may contain the data to build some/all of the aggregate objects
  3. Multiple Objects – a ResultSet may contain the data for many objects in one row

Challenge #1 can occur whenever inheritance is used, Challenges #2 and #3 are linked with improving performance (though I’m finding it difficult to imagine a scenario for #3 right now).

As well as the challenges there are some improvements that may be made:

  • The current BeanProcessor mapping works with the JavaBean standard – getColumnName/setColumnName will map to columnname in the database. A solution that also works with Annotations and/or Object fields would be helpful.
  • Automatic name mapping of the object ’standard’ (camelCase) to the database ’standard’ (under_score) would also be helpful.
  • Since I’m a user of the Spring framework, something that I can use as a default RowCallbackHandler

So what would my ideal mapping solution look like in operation?

I would have JDBC DAO that maps a variable ResultSet automatically to a set of target classes…

public interface JellyDAO {
  public Jelly find(Long jellyId);
  public List<Jelly> find(FindRequest findRequest);
} 
public class JellyDAOJdbc implements JellyDAO {

  List<Class> targetClasses;
  List<Class> aggregateClasses;
  BeanMapper beanMapper;

  public jellyDAOJdbc() {
    // The ResultSet may contain data for 
    // any of the following target classes.
    targetClasses = new ArrayList<Class>();
    targetClasses.add(JellyBean.class);
    targetClasses.add(JellyCup.class);
    targetClasses.add(JellyCupAndSpoon.class);
    // The Target classes may contain the following classes
    // that may also come across in the ResultSet.
    aggregateClasses = new ArrayList<Class>();
    aggregateClasses.add(JellyAttribute.class);
    aggregateClasses.add(JellyCompany.class);
    //
    beanMapper = new BeanMapper(targetClasses, aggregateClasses);
  }

  private Jelly processResultSetRow(ResultSet resultSet) {
    // Map the ResultSet to a Jelly class.
    Jelly jelly = (Jelly) beanMapper.toBean(resultSet);
    return jelly;
  }

  private List<Jelly> finder() {
    // Obtain a ResultSet..

    // Loop through result set..
    {
      jellyList.add(processResultSetRow(resultSet));
    }

    return jellyList;
  }

}

The Jelly domain model might be something like…

public abstract class Jelly { 
  Long jellyId; 
  String jellyType; 
} 
public class JellyAttribute { 
  String name; 
  String value; 
} 
public class JellyCompany {
  String companyName;
  String address;
}
public class JellyBean extends Jelly { 
  String targetMarket;
  BigDecimal weight;
  JellyAttribute flavour; 
  JellyAttribute colour; 
  JellyCompany company;
}
public class JellyCup extends Jelly {
  String productName;
  BigDecimal volume;
  JellyAttribute shape;
}
public class JellyCupAndSpoon extends JellyCup {
  String spoonMaterial;
  BigDecimal spoonLength;
}
public class JellyRepository {
  public Jelly find(Long jellyId);
  public List<Jelly> find(FindRequest findRequest);
}

I would also have a database view or stored procedure with the following signature…

jelly_id, jelly_type, target_market, product_name, 
spoon_material, spoon_length, cup_volume, flavour_name, 
flavour_value, colour_name, colour_value, company_name, 
company_address, shape_name, shape_value

Sep 06 2006

The Four Horseman of the Data Apocalypse

Tag: warren @ 10:05 pm

The data was not longer required.

First came the horseman named Delete, he said in a thunderous voice, “You data will soon be gone.”

The user replied, “But what if I want to see it again.”

Next came the horseman named Trash, she said, “You data will be gone soon, but if you sign up for this simple insurance policy, it can be brought back – if you invoke the policy quickly.”

The user replied, “But I may want it at any time in the future.”

Next came the horseman named Archive, he said philosophically, “You data will not be available, except when it is.”

The user replied, “But I want it to be available at any time.”

Next came the horseman named Retire, and she said, “You’ve put effort into your data, you don’t want it disappearing. It may not be as active as it once was, but with good planning you may keep it working for you always.”

The user smiled and was happy, “Yes, I want them all.”


Jul 27 2006

Repository or DAO?

Tag: warren @ 2:03 pm

A few months ago, while considering the design of a new project…the issue occuping my thoughts was in the differences in possible design technologies. Two directions: one a relational database, two an object model. Why was it an issue? How are the designs kept in sync? Where does a developer turn to first if they want to make a change to the core design of an application that is expressed in many places (at least the relational database and the objects)?

So, I considered the idea of a ‘domain model’, turned it around a few times and thought it could be a good thing. Not only would it be the common language between various technical areas, if it was expressed in business terms, the testers and business people could also be involved in a discussion around it, or use it. Convinced this was something good, I Googled ‘domain model’ and found that 1. yes it was a good idea and 2. someone had written a book about it. I ordered Domain Driven Design by Eric Evans and have been reading through it.

Good book (pity it goes for the fashionable ‘xxx driven xxx’ title, which also sounds confusingly similar to Model Driven Development). It has clarified many things for me, and supported other thoughts, like the ‘Ubiquitous Language’ pattern… “A project faces serious problems when its language is fractured.”

Many standard patterns for domain components are listed and discussed in the book. One pattern is called a Repository. It troubled me because elsewhere in the world of development there is a DAO (Data Access Object) pattern. Why was I concerned? Because the two patterns initially seemed to be the same, which should be used? This is what I understand now.

Repository Responsibilities

  • Provide a common language to all team members (including business representatives to whom DAO would need constant explanation)
  • Provide a higher level of data manipulation – something that may be common to the data regardless of how it is retrieved.
  • Provide a mechanism to manage existing entities (where a Factory might create you a new one).

DAO Responsibilities

  • Provide data access. Perhaps you could say it is about persistence strategies, underneath a DAO interface there is one or more implementations – eg JDBC, Hibernate.

Example: the domain model might represent plants, one entity may be a Tree which has an associated TreeRepository.

TreeDomainModel

Implementation 1:

There is a database table in the background with TREE_ID, LATITUDE, LONGITUDE, HEIGHT columns. Perhaps we code TreeRepository as an interface which a JDBC DAO implements.

Implementation 2:

We now access to a location Service which stores the location of things. We also have a database table with TREEID, LOCATIONID, HEIGHT columns. Perhaps we implement a TreeRepository that accesses both the location service and JDBC DAO to construct a full Tree object.


Jul 25 2006

Our new robot – what next?

Tag: warren @ 7:11 pm

We recently bought a roomba robot vacuum cleaner…it gives us back some time. One of the best bits of technology that we have purchased lately.

It started me thinking about the mid nineties when I signed up with my first ISP and jumped into the internet and the early eighties when my father bought home a new Apple II computer.

Wonder what consumer robots will be like in 10 years?

Also, I wonder what will be ‘introduced’ in the next 10 year hop – nano-bots, bio-tech?


Jun 13 2006

How Unit Testing is changing my coding style

Tag: warren @ 1:51 pm

One of the books that I am reading at the moment is “First Things First”, it mentions a cultural difference (yes blatant generalisation) between east and west along the lines of:

  • West – if it ain’t broke, don’t fix it
  • East – kaizen (continuous improvement).

This idea probably encapsulates the biggest change unit tests have made to my approach. I will now design and code with confidence that I can improve the code at any time.

That idea allows me to start without (too much) future thought as to what will be required – great for someone who makes use of intuition to program. I have often spent considerable non-programming time proving to myself and others that an idea will work. What does having a unit test have to do with this? It provides a framework where change is expected – sometimes an intuitive idea just does not work. I can rely on tests to prove the idea, or not. A schedule of work can be broken into smaller pieces, each of which I can focus on individually. The individual focus effect provides a productivity gain. Instead of building the mental castle of an entire source code structure, I can concentrate on the unit in front of me. (If there is a side effect because of what I do, it will be reported when the tests are run – note that I include integration testing in my idea of a ‘unit’ test framework.) Though I was always willing to make cosmetic code improvements to something that I had personally written, I would regularly reach a point where I would be hesitant. Now, things like cosmetic changes to provide better code understanding, and design changes to better suit the application are done (almost) without hesitation. It is possible to ensure that the code still works even when I make major structural changes.

The cost in using a unit test framework is of course in writing and maintaining the tests. However, the way I have always worked, is code a bit, test a bit, code a bit more, test a bit more, etc. So there has always been a testing cost in what I write, it is just that now the test has also become a deliverable. There could probably be a debate on which approach costs more – possibly over a project lifetime they may be the same? Plus I like the idea that I can code a bit, test that bit, code a bit more, test it all, code a bit more, test it all, etc.


May 31 2006

CSS for a complex web form

Tag: warren @ 4:57 pm

I just spent some time styling a complex(?) web form with CSS. Some basic requirements:

  • Any number of input elements on each form line
  • The first label on each line to be a fixed width – like a column
  • Able to indicate required/optional/readonly fields
  • Able to move things around fairly easily – particularly after a few months of not touching the code (and in my opinion counting td/tr and cell spanning is not easy)
Complex Form Example

Steps to achieve this result:

  1. First enclose each form line with a div (class=row).
  2. Add an indicator to the first label element on each row (class=first).
  3. Enclose the relevant pairs – label and input/select/etc – with a span (class=optional/readonly/required).
  4. Add CSS to suit the requirements.

So, it took a bit longer than it should have…

The label element does not have a width property unless you float it.

When floating the ‘first’ label, the text changed vertical alignment relative to the other labels in the line. I could fix it by applying a margin to the ‘first’ label, but when I changed font-size (or the text size via the browser) the alignment was out again.

The problem seemed to be caused by floating the label for for a font-size smaller than height of the input box. The simple fix seems to be in adding a line-height to all labels.

You can see the result (HTML + CSS) of this excursion here.


May 22 2006

Ammunition against ‘best practice’

Tag: warren @ 8:38 am

I was asked this week for help in responding to a person who continually supported their approach by calling it ‘best practice’.

Best Practices: Who Says?

“The fact is that we really don’t have a mechanism to formally establish best practices in the software development community.”

“… I’ve begun restricting use of the term ‘best practice’ in articles that we publish in IEEE Software. In most cases, ‘good practise’ or ‘effective practice’ serve as acceptable substitutes.”

Best Practice

A completely fatuous concept based on two dangerous assumptions:- a) someone else knows what’s best for you b) you don’t have to think about your context.

No Best Practices

“Go ahead and follow your favorite practices. Just don’t preach that the rest of us must follow them, too. Keep your process standards to yourself. If you want to make a suggestion, make one that takes context into account.”

Useful Practice

  • BestPractice – someone else knows what’s best for you
  • UsefulPractices – you are stuck deciding what’s best for you. It’s your butt, after all.
  • BestPractice – you don’t have to think about your context.
  • UsefulPractices – you are the one in your context. So pick what works for you there.
  • BestPractice – you can have success by doing these things without understanding anything about them.
  • UsefulPractices – only through understanding of the practices and their principles do you stand a chance of succeeding.
  • BestPractice – is an exclusive practice. These are best. We know best. The work is in selecting out.
  • UsefulPractices – is an inclusive practice. These seemed to work. We have some ideas. The work is in selecting in.

May 14 2006

How do you to-do?

Tag: warren @ 6:10 pm

Let’s start with a simple definition of a ‘to-do’, how about – a task that you cannot complete right now.

The issue is what you will do with that task. Basically, you want to forget about it now, but remember to start the task when you have the required time, resources and inclination. This implies that the being a reminder mechanism is the main objective of a to-do.

So, the basic to-do scenario might be:

  1. Open up your to-do list
  2. Add the task to it
  3. Return to whatever you were doing
  4. Sometime in the future start doing the task

An important concept was just introduced! Many to-do tasks make up a to-do list. Your to-do list may spread across many formats:

  • Memory
  • Physical prompt
  • Paper
  • Software
  • An assistant

Time to remember the primary objective – how to remember when to do something. Notice that all of the to-do list formats have a direct method of notification:

  • Memory – It’s Monday, back to work, I have to bring that book in
  • Physical prompt – I need that book at work, I’ll put it by the front door
  • Paper – I’ll add a note to my morning checklist to bring that book in
  • Software – I’ll email a reminder to my home email because I need that book tomorrow
  • An assistant – Hey slave, go to my house, pick up this book that I want at 2pm

All of them work, all of them have issues…

To-do List Advantages Problems
Memory Portable No/low overhead ‘entering’ a new task Forgetfulness Stress Basically unreliable
Physical Prompt Effective set and forget method Typically the prompt is what you need to complete the task Other people can interfere with the prompt Probably not scalable Creates clutter Only for suitable tasks (eg does not work for “Take the car to the car wash”)
Paper Can represent any task Well supported by to-do systems Configurable in any way you want to change it Relatively inexpensive Portable Requires manual checking Can involve significant management Requires an organised personality Must be carried around to be portable If you lose it, it is gone forever For complex systems, you must understand the system
Software Can represent any task Well supported by to-do systems Reminders can be automated Much of to-do system theory can be built into the software Can create backups to guard against information loss Can be inexpensive Portable (on a Mobile device) Mobile device must be carried around to be portable May not support features that you want (difficult to add those features if you want them) Can be expensive Requires access to a compliant device for the reminders to work
An assistant Task management can be sub-contracted to other people Very Expensive

Note that we use a combination of most to-do list types – even when we attempt to use only one. Recognising this and working with it, rather than against it, can help you make peace with your personal to-do ’system’. You can allow yourself to use the best to-do list for the situation rather than fit all situations into a prescribed to-do list.

One common problem with to-do lists is that few of them cope effectively with lots of tasks. As soon as a to-do list is filled wtih many tasks, it becomes more complex to manage. Strategies to cope with many tasks include:

  • grouping tasks into projects
  • giving tasks tags or context – typically location, priority and scheduled date/time.

What other things can you do with a to-do list?

As time is a limited resource, and we tend to have more to-do than available hours, you can use a list to prioritise what you will target for achievement.

What can you do with a to-do when you’re done?

You can report on what you have been doing (perhaps to your boss).

Basically, a done list provides you with a feedback loop. Just like recording money expenditure helps with budgeting, a completed to-do list can assist you in examining where your time is going. You can better balance your life activities.

The temptation though, is to track more with a to-do list than necessary, just to improve the quality of the feedback.

Actually, this function could better be called a task diary. Completed to-do items happen to be the main (probably only) input to the task diary for most of us. However, the value of the task diary is limited by how many done tasks are added to it. A better approach may be to consider more than just completed to-do items as input to your task diary (eg a direct automatic daily entry for making and eating breakfast).


May 12 2006

Steps to open source a project

Tag: warren @ 3:50 pm

Recently I open sourced the MIGRATEdb project. Here are a few notes about the trip.

A home for the project

To open source a project, first you need to choose where to locate your project. One way would be to self-host the project and source code, however there are a number of service providers out there who will manage your source code, Subversion/CVS repository, track issues, allow downloads, etc. I chose SourceForge. So I had to create a user account and then setup the SourceForge project.

Project Setup

When setting up the project there a few important things to consider – like the name of the project. Why MIGRATEdb, well the name provides some indication of the projects purpose and it conjured a couple of interesting visuals…a bee with suitcases (a migrated-B)…the ‘db’ part is vertically symmetrical.

To open source a project you must choose an open source license to use. There are many of them, to ease the brain strain they can be grouped into two main types: academic and reciprocal. Basically, an academic license allows anyone to use the project in any way they want, a reciprical license requires any derivatives, that are released and distributed, be provided under the same license. MIGRATEdb was open sourced under the BSD license an academic style license.

Project Registration with Host service

Next comes the actual registration process. For SourceForge there were a number of questions to answer (like how the software is classified, what is a short description of the software’s purpose, …), then someone will actually look at the project request and hopefully approve it.

Bring the project to life

These first steps were all performed with SourceForge tools:

  1. Upload the source code to the Subversion/CVS
  2. Create a release, for MIGRATEdb there are two release files – the full project and an ‘executable only’ distribution.
  3. Create some documentation – for MIGRATEdb this is a simple web page.

Promote the project

To raise interest in the project you can promote it to the targeted user community, blog about it, tell all your friends, etc.


Apr 23 2006

What is my context?

Tag: , warren @ 6:42 pm

Personal Context. It can be used to enhance software interaction with you. Time has been the big one so far. You tend to know when you are fairly cheaply. Time can be used in software fairly reliably – problems are mostly encountered when you personally travel or you communicate with someone in a different time zone.

What else could work as context about you?

  • Email address – identification and contact mechanism. It is distinct, regularly used as a login name, but suffers a little because it changes.
  • Biology (fingerprints, dna, eye scan, heart rate, blood pressure) – identification, state of health, what you are doing (eg brain waves indicating sleep/dreaming).
  • Written signature – identification.
  • GPS – answers the “where am I” question. Could this be the next big thing in Personal Context?

Software that knows where it is. How do I move from here to there? Navigation systems do it and have already been implemented all over the place. Sports watches already use GPS information to provide speed and distance for the runner/cyclist/sailor.

I have a personal interest in building a better to-do list (perhaps by creating some software). How about combining your pda/mobile phone, with GPS, with location and time aware to-do software? Now I can have software that automatically:

  • only shows tasks relevant to work when I am in the office
  • reminds me to buy bread when I walk near the grocery store
  • reminds me to buy a birthday present when it is a week to a friends birthday and I am near a shopping centre
  • reminds me of something that I want to tell a friend when I visit them

What about the actual mobile phone software – I want calls from work to only vibrate the phone when I am at home, but to ring when friends call.

GPS, with its growing and cheaper hardware base, looks very interesting in combination with software that can make use of my personal context.


« Previous PageNext Page »