Database Migrations Using DBXExpress Metadata

by John 12/19/2007 5:56:00 AM
Overview

I recently spent some time looking at Ruby On Rails. One of the many things about Rails that caught my interest was the way that schema changes to databases are performed using Ruby. I think it was this post by Steve Shaughnessy about the new dbExpress 4.0 MetaData where it occurred to me perhaps I could use the DBExpress MetaData to do my something similar in Delphi.

I had several goals in mind:-
  • Produce a repeatable series of steps to upgrade database schemas.
  • Stay away from having to remember SQL syntax.
  • Automate the upgrade process
  • Target multiple db servers


Before presenting my solution, I would like to thank Leonel Togniolli, CodeGear R&D for helping me with how to go about calling the DBExpress MetaData API.

Project Layout

I have bundled everything together in a projectgroup which contains five projects.
Moshine.Migration.Framework.dll - Contains most of the core code
Moshine.Migration.Framework.Tests.dll - Contains some unit tests
Moshine.Migration.Test.dll - is an example of a database migration
Moshine.MigrationRunner - Is a VCL.Net application to execute migrations
Moshine.MSBuildMigration.dll - Contains a collection of MSBuild tasks that can be used from the IDE or standalone from the RAD Studio Command Line

Example

A database schema generation or migration is composed of multiple steps. A step is the equivalent to a Delphi class.  A step can perform such actions as creating tables, adding columns, adding indexes, adding foreign keys, and executing sql. Sql could be used to add stored procedures and insert domain data into tables.

So if you take a look at Moshine.MigrationTest.dll, this contains seven units which each contain a Delphi Class. So after running the migration the database would be at version 7. For example this is the declaration for the first step

[Migration(1)]
TBookMarksFirstMigration=class(TMigration)
public
    procedure Up();override;
    procedure Down(); override;
end;

Migrations can go backwards and forwards so I have Up and Down methods. The idea is that to create a step you create a new class derived from TMigration and decorate with an attribute which describes which step this is. Migrations must start at 1 and be sequential.

If you take a close look at the Up method in TBookMarksFirstMigration, its creating a table called Bookmarks using the DBXExpress metadata API. After table creation the column ID is set as the primary key.

The corresponding Down method for this class would drop the table.

TMigration


All migration steps must be derived from TMigration. This base class is in Moshine.Migration.Framework in Moshine.Migration.Framework.MigrationBase.pas. I have provided a number of protected methods that Migration developers can call

procedure AddColumn(tableName:string;columnName:string;dataType:integer);
procedure RemoveColumn(tableName,columnName:string); procedure AddIndex(tableName, columnName: string; indexName: String);
procedure RemoveIndex(tableName, columnName: string; indexName: String);
procedure AddPrimaryKey(columnName,TableName:string); procedure AddForeignKey(columnName,tableName,primaryTableName,primaryColumn:string);
procedure RemoveForeignKey(columnName,tableName,primaryTableName,primaryColumn:string);

There are also 2 properties

property Provider:TAdoMetaDataProvider read FProvider write FProvider;
property Connection:DbConnection read FConnection write FConnection;

Connection is a database connection but Provider is the most important. Its a TAdoMetaDataProvider object and is essentially the DBXExpress metadata. The protected methods I supply are basically calling methods in the metadata API. Most of the time I would expect that you would call the helper methods but you can use use the Provider or Connection to execute raw sql.

MSBuild Setup


I have been using MSBuild target files to define targets that call the various tasks. The Moshine.MigrationTest.dll project contains one such targets file called TesTargets.Target

The MSBuild tasks are driven by two properties, a connection and assembly that contains your migration.

Moshine.MigrationTest is a sample migration. The project contains an app.config

This contains your connection specifications. For example mine contains

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <connectionStrings>
        <add name="SomeConnectionString" connectionString="database=C:\develop\moshine\Migration\Moshine.MigrationRunner\App_Data\migration.jds;user=SYSDBA;password=masterkey;host=localhost;protocol=TCP;create=true" providerName="Borland.Data.BlackfishSQL.RemoteClient"/>
    </connectionStrings>
</configuration>

a single Blackfish connection called "SomeConnectionString"

The other item that must be added to a project are the MSBuild tasks. I have added TestTarget.targets to Moshine.MigrationTests.dll

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrateDownTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />
    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrateUpTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />
    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrationTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />
    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrationSchemaVersionTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />

    <PropertyGroup>
        <MigrationAssembly>C:\develop\moshine\Migration\Moshine.MigrationTest\bin\Moshine.MigrationTest.dll</MigrationAssembly>
        <ConnectionName>SomeConnectionString</ConnectionName>
    </PropertyGroup>

    <Target Name="Migration">
        <TMigrationTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />
    </Target>

    <Target Name="Up">
        <TMigrateUpTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />
    </Target>

    <Target Name="Down">
        <TMigrateDownTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />
    </Target>

    <Target Name="SchemaVersion">
        <TMigrationSchemaVersionTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />
    </Target>

</Project>

The contents is above. I have setup two properties and four targets. The two properties define the Connection to use and the assembly that contains the migrations that will be run by the MSBuild targets.

You should now be able to open a RAD Studio Command Prompt. It you change to the Moshine.MSBuildMigration directory you add run the migrations. For example

msbuild TestTarget.targets -t:up
msbuild TestTarget.targets -t:down
msbuild TestTarget.targets -t:SchemaVersion
msbuild TestTarget.targets -t:Migration

You can also run the migrations from the IDE project menu as well. In the project manager select TestTarget.targets in the Moshine.MigrationTest project, right click and select the targets menu item. Listed as submenu items of the targets menu will be the same four targets as above, as can be seen below.



The SchemaVersion targets displays the database schema version.
Migration executes all steps defined in the assembly.
Up and down step up and down the migration.

So say I have my database at database schema version 4 and I execute the migration target. Steps 5, 6 and 7 will be performed.


Running a Migration



The Migration Framework looks for a database table called Migration_Schema to determine the database version and then looks at the steps provided in the metadata assembly to work out the steps to run.

If you issue the Migration target and the number of steps doesn't match the schema version the appropriate steps will be run.

You can also use the Up and down targets step up and down the migrations. MigrationSchemaVersion outputs the database schema version.

Moshine.MigrationRunner

I have also provide a VCL.Net application which duplicates the MSBuild functionality. This is configured with app.config. An example of which can be seen below.

Note(s)

To be able to build the projectgroup you will need the December 2007 Rad Studio Update installed.

The Moshine.MSBuildMigration.dll assembly relies on Moshine.Migration.Framework.dll, so make sure that both assemblies are located wherever you set the using task.

i.e

<UsingTask TaskName="Moshine.Migration.MSBuild.TMigrateDownTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />


In theory you should be able to perform migrations on any database that CodeGear provides MetaData for. I have done some testing with AdoDbx against MSSQL, Interbase and MySql. Most of the time I have been using the Blackfish native providers.

If this interests you, I have placed the code in codecentral at this location. Absolutely any feedback is welcome.

Currently rated 3.3 by 6 people

  • Currently 3.333333/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Tags:

VCL.Net | CodeGear | Blackfish | migration | Delphi | msbuild | Database | dbx4

Starting WCF Programming in Delphi 2007

by john 11/7/2007 8:42:00 AM

I have just started looking into Windows Communication Foundation.

Pawel Glowacki created a really nice article on the CodeGear Developer Network titled Introduction to WCF Programming in Delphi.

I'm particularly interested in web development and I noticed that there is a WCF Service Template in Visual Studio 2005, so I have converted it to Delphi.

I have uploaded the template to CodeCentral.

If you are not sure how to use the template, I have a blog post about templates and how you install them,

Once installed you can now create a new WCF Service application

 

You should end up with a new ASP.Net  Web Application with a single WCF Service.

 

Assuming that you have already installed Microsoft dotnet v3.0, you can now build and run the application.

The service description should now appear in a browser.

 

To exercise the web application you need to do the following:-

1) Create a new VCL.Net application

2) Add a reference to the System.ServiceModel assembly

3) Add

C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.0

to the compiler search path

4) Your code might look something like

uses
  System.ServiceModel, Service, System.ServiceModel.Channels;

{$R *.nfm}

procedure TForm1.Button1Click(Sender: TObject);
var
  aEndpointAddress: EndpointAddress;
  aBinding: WSHttpBinding;
  aFactory: ChannelFactory<IMyService>;
  aChannel: IMyService;
  someValue: string;
  value: DataContract1;
begin

  try
    aEndpointAddress := EndpointAddress.Create('http://localhost:8080/WCFServiceWebApplication1/Service.svc');
    aBinding := WSHttpBinding.Create;
    aFactory := ChannelFactory<IMyService>.Create(aBinding, aEndpointAddress);
    aChannel := aFactory.CreateChannel;
    try
      someValue:=aChannel.MyOperation1('John');
      ShowMessage(someValue);

      value:= DataContract1.Create;

      value.FirstName:='John';
      value.LastName:='Moshakis';

      aChannel.MyOperation2(value);
      ShowMessage(someValue);

    finally
      IChannel(aChannel).Close;
      IDisposable(aChannel).Dispose;
    end;
  except
    on E:Exception do
    begin
      raise;
    end;

  end;

end;
 

 A word of warning, if like me you use Cassini as your web server, I think you need to upgrade to the latest version. Below is a screenshot of the application running with the version of Cassini supplied with Delphi 2007.

 

 

Please read my blog post about upgrading Cassini.

Comments welcome 

 I also see that Pawel is doing a presentation at CodeRage II titled "Delphi for .NET WCF Programming: Deeper Dive", I'm looking forward to listening to that,

Currently rated 3.0 by 2 people

  • Currently 3/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Tags:

ASP.Net 2.0 | WCF | Delphi | VCL.Net

Powered by BlogEngine.NET 1.3.0.0
Theme by Mads Kristensen

About the author

Name of author John Moshakis
I'm a software developer living in Toronto..

E-mail me Send mail

Calendar

<<  November 2017  >>
MoTuWeThFrSaSu
303112345
6789101112
13141516171819
20212223242526
27282930123
45678910

View posts in large calendar

Pages

    Recent posts

    Recent comments

    Authors

    Tags

    Don't show

      Disclaimer

      The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

      © Copyright 2017

      Sign in