Monday, June 28, 2010

What to choose to achieve loading speed

It is important to use the right tool for the job. As a developer there is often the choice between to use a technology that let you do anything at the cost of you must also take care of everything. Or use a framework that take care of most details. The level of abstraction is raised. Bold for Delphi is such framework.

The header of todays post says Loading speed. Performance is often important in applications. Here is a simple list of techniques of show data from a database with comments about speed.

  1. Using OCL with all dependent objects in object-space (RAM) and PS set to False. I used to call this warm objects. In my personal experience there is nothing that can beat that in speed. The GUI is very fast updated.
  2. Handwritten SQL versus Bold is like assembler versus C or C++. When written properly it can be very fast. But it is also easy to write slow SQL with sub-queries, introduce unnecessary complexity etc.
  3. Using OCL with dependent objects not in object-space (RAM) and PS set to True. This is cold objects. In some cases it can have equal performance as well written SQL. Note there is some limitations of the expression as derived attributes and links cannot be used. The result must be a collection of Bold objects.
  4. Using OCL with all dependent objects not in objects in object-space (RAM) and PS set to False and prefetch is used to load objects. See the explanation what prefetch is.
  5. Using OCL with all dependent not in objects in object-space (RAM) and PS set to False and prefetch is not used. This is lazy fetch. One SQL call is generated for every object loaded and this can be a bit slower.
One thing to be noted is that the database cache has big impact of the performance. The second time the same SQL is executed the data is already in the cache and is delivered faster.

When Bold load data and PS is set to False Bold evaluate and generate the needed SQL to load data in object space. The next time Bold need the same object it is already in memory and the speed gained is very high. It is number one at my list above. The problem is when new data is to be loaded all the time.

Bold contains some methods to avoid lazy fetch, EnsureObjects and FetchLinksWithObjects. The problem is then the model grow an by just reading a derived member may start a chain of lazy fetches that could be hard to anticipate.

One solution we at Attracs use with success is prefetching. When loading bigger datastructures from database it is often many instances of the same class. It is much more effective to load many instances in one SQL call compared to one SQL call per instance. We use some kind of prefetch language to archieve this.
A prefetch expression can look like this:

(activeCustomers, (activeOrders, invoice))

The precondition in this fabricated case is class CustomerPool that have a multilink ActiveCustomers that have the class Customer in the other end. The Customer have a multilink to Order class by the multilink activeOrders. And the Order class have a link to Invoice class. by doing a prefetch on this then all Customers is loaded in one SQL and the same with Order and Invoice class. The result is better user experience.

Some people may think that the syntax of this prefetch language is a bit funny. It should be noted that prefetch is not part of the Bold framework (yet). It is an in house technique to get better performance.

Now we also have a method that generate a prefetch expression from an OCL expression. This make it much easier as the prefetches can adapt from changes in model. The result is more dynamic and less hard coding!

Saturday, June 12, 2010

A faq about using Bold for Delphi

A FAQ about Bold for Delphi
  1. What is the requirements to start using Bold ?
  2. What is UML ?
  3. What is ORM ?
  4. What is OCL ?
  5. What is a derived member ?
Q: What is the requirements to start using Bold ?
A: Currently Bold is only available in 3 editions, all in obsolete versions of Delphi:
  • Delphi 7 Architect.
  • Delphi 2005 Architect
  • Delphi 2006 Architect
  • Delphi 2007 using package from Delphi 2006
So Bold can be used with D2007 if the package was compiled with D2006 as they are binary compatible.

Q: What is UML ?
A: UML (Unified Modeling Language) is a standardized general-purpose modeling language. Bold use MDA (Model-driven architecture) meaning that a change of the model is the first step in development. Then that change is applied on the database. Then the code may be changed to adapt to the previous model changes. In UML you can specify relations between classes as singlelinks (on to one) or multilinks (one to many). Each class can have attributes of basic datatypes and methods attached.
See also UML on Wikipedia

Q: What is ORM ?
A: ORM (Object-relational mapping) means to convert relations in a traditional SQL database to objects in memory that can be used with a objectoriented language. It is of course possible to change objects and store them in the database. This makes programming much easier as it is not needed to write SQL to read and writa data from the database. It is much easier to write complex expressions if you don't need to use SQL.

Compare this by OCL ?:

PlanMission.allinstances->select(created > #2010-02-04).created

and in SQL:

SELECT created FROM PlanMission WHERE (PlanMission.Created > '2010.02.04')

See also ORM on Wikipedia.

Q: What is OCL ?
A: OCL (Object Constraint Language) is a language that is used a lot in Bold. It has no sideeffects meaning that there is no way of changing objects. All operations is read-only. It is easy to work with a list of objects and filter a certain property or type.

You can build an OCL expression in designtime by a dialog that contains a context sensitive list of keywords.

See also OCL on Wikipedia.

Q: What is a derived member ?
A: A derived member is typically not stored in the database. It is transistant and exists only in Bolds objectspace in RAM. For example if you have 3 classes Invoice, InvoiceRow and Vat.
The InvoiceRow have the attribute NetValue and a singlelink to Invoice. Invoice have a single link to Vat and a multilink InvoiceRows. Vat have an attribute VatValue.
Now InvoiceRow can have 2 derived attributes VatValue and TotalValue. Invoice can also have 3 derived attributes NetValue, VatValue and TotalValue. See the model.
UML model
So class Invoice have 3 attributes, InvoiceRow also 3 and Vat have 1. Only 2 of those 7 attributes are stored persistent in the database, The remaining 5 is marked as derived with a / before the name.

We can define the value for a derived attribute in a method or in OCL. Often OCL is simpler and safer. In code you have to take care of subscribing other attributes and objects in case they change. In OCL this occurs automatic.

OCL expressions for derived attributes:

  • InvoiceRow.vatValue: netvalue * mainInvoice.invoiceVat.vatValue
  • InvoiceRow.totalValue: netValue + vatValue
  • Invoice.netValue: invoiceRows.netValue->Sum
  • Invoice.vatValue: netValue * invoiceVat.vatValue
  • Invoice.totalValue: invoiceRows.totalValue->Sum
Derived attributes have the property that they are recalculated every time a dependent object change and that attribute is read.

NOTE: The recalc does only happen if the derived attribute is read.

In this case if one of the Invoicerows netValue change and this InvoiceRows vatValue is read it notice the change and recalc. But totalValue is not recalced until it is read. When Invoice.totalValue is read it will be recalced if any of the other attrbutes have changed value. If nothing has changed the attribute value is cached and the calculation will not happened.

This behaviour save CPU time specially for complex expressions. Compare if those attributes would be methods that do the calculations every time they where called. And of course the chain of derivations can be arbitrary long and complex.

The first post about Bold

So this blog is about Bold for Delphi. There is a lot of developers that knows Delphi and using it for development. But I am still amazed that so few programmers are not aware of modeldriven development. I started to work with Delphi and Bold Sep 2005. Soon I realize the potential with this language and framework. Here you have the power of object orientation with Delphi and RAD development. That in combination with object orientation also to the persistence or the database.

Traditionally programmers has used SQL to communicate with a database to load and store data. Sure it works, but when the complexity grows it is easy to get a nightmare when fixing bugs or adding new features. By using Bold for Delphi as framework the productivity is increased a lot (I have heard 10 times as effective). You handle data an the database and the model as objects and this way you can take much more complex situations compared to the traditional way.

Bold use an UML model and OCL to read data from database. Maintenance is easy, just change the model, apply the generated database script to the database and start coding the changes. You don't need SQL, but you can of course still use it if you really wants.