Wednesday, July 30, 2008

Common Base Class for LINQ to SQL

Language-Integrated Query (LINQ) is a set of features in Visual Studio 2008 that extends powerful query capabilities to the language syntax of C# and Visual Basic. As a part of LINQ, LINQ to SQL provides a run-time architecture for managing relational data as objects. To some extent, it equals to ORM tool or framework such as NHibernate and Castle based on .NET framework. It becomes our preferred choice gradually when we want to access the database.

In LINQ to SQL, all variables in the Data Model of a relational database can be strongly typed which provides the benefit of compile-time validation and IntelliSense. We can fetch the data from the database using query expression (it includes query syntax and method syntax.)

However, the strongly typed feature is not conducive to abstract the common logic of data operations so the developer has to define a specific class to handle the entity object. It results in a large number of repeated codes. If we can implement the base class which encapsulates the common operations such as Select, Where, Add, Update and Delete, it will be useful for N-Tier application.

Fortunately, the generic type will help us to achieve our goal. We can invoke a method called GetTable() in DataContext of LINQ. For example, we can implement the Where method which accepts the Lambda expression to find the result we want to get:

public IList<TEntity> Where(Func<TEntity, bool> predicate)

{

    InitDataContext();

    return m_context.GetTable<TEntity>().Where(predicate).ToList<TEntity>();

}

 

It’s simple. Even, we can expose the method which accepts the condition clause using Dynamic Query:

public static class DynamicQueryable

{

    public static IQueryable<T> Where<T>(this IQueryable<T> source, string predicate, params object[] values)

    {

        return (IQueryable<T>)Where((IQueryable)source, predicate, values);

    }

 

    public static IQueryable Where(this IQueryable source, string predicate, params object[] values)

    {

        if (source == null) throw new ArgumentNullException(“source”);

        if (predicate == null) throw new ArgumentNullException(“predicate”);

        LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, typeof(bool), predicate, values);

        return source.Provider.CreateQuery(

            Expression.Call(

                typeof(Queryable), “Where”,

                new Type[] { source.ElementType },

                source.Expression, Expression.Quote(lambda)));

    }

}

public IList<TEntity> Where(string predicate, params object[] values)

{

    InitDataContext();

    return m_context.GetTable<TEntity>().Where(predicate, values).ToList<TEntity>();

}

 

Of course, the query function is not a problem because we don’t need the properties of the entity when we invoke these methods and don’t care about the composition of Lambda Expression.

The key issue is how to update or delete the record of data table. You know, we must fetch the entity which is going to be operated before updating or deleting it. The keyword to search is often its identity. Furthermore, Object Identity and Change Tracking in Data Context need the object’s identity to track the changing of it. Usually, we would add the Id column which is identity or Guid for each table. Then we can fetch the entity object according to it:

    public void Update(Employee employee)

    {

        LinqSampleDataContext context = new LinqSampleDataContext();

 

        Employee emp = this.Where(e => e.EmployeeID == employee.EmployeeID);

        emp.FirstName = “First Name”;

        emp.LastName = “Last Name”;

 

        context.SubmitChanges();

    }

 

Obviously, we don’t know the entity’s properties if we use the generic type such as TEntity like above. Without the keyword how do we associate the changes with the existing record? LINQ introduces the Attach method which can attach the changed object to the context. There are three overloaded versions as below:
Attach(Object entity): Attaches an entity to the DataContext in an unmodified state;
Attach(Object entity, bool asModified): Attaches all entities of a collection to the DataContext in either a modified or unmodified state.
Attach(Object entity, Object orginal): Attaches an entity to the DataContext in either a modified or unmodified state by specifying both the entity and its original state.

Attach method is used to associate the deserialized entities to a new instance of a DataContext. However, we can associate the entity in one DataContext to another DataContext by using it. In update or delete case, this method is very useful. For example, we want to update the record according to a changed entity from another DataContext:

    public void Update(TEntity changedEntity)

    {

        InitDataContext();

 

        try

        {

            m_context.GetTable<TEntity>().Attach(changedEntity, true);

            m_context.SubmitChanges();

        }

        catch (ChangeConflictException)

        {

            m_context.ChangeConflicts.ResolveAll(RefreshMode.KeepCurrentValues);

            m_context.SubmitChanges();

        }

    }

 

It looks good, but not enough. We can’t attach a changed entity unless the entity has a TimeStamp column in the corresponding table. Therefore, we should add a column whose type is TimeStamp in SQL Server 2005, or set the IsVersion property to true for Id property in LINQ to SQL Designer. My advice is to create a TimeStamp column for your datatable, and it will improve the performance because it won’t check all columns whether they are changed during handling concurrency.

In fact, we can pass the original entity with generic type also. The only problem is how to pass the values you want to change. The solution to this problem is using Action delegate. The code snippet is as below:

    public void Update(TEntity originalEntity, Action<TEntity> update)

    {

        InitDataContext();

        try

        {

            m_context.GetTable<TEntity>().Attach(originalEntity);

            update(originalEntity);

 

            m_context.SubmitChanges();

        }

        catch (ChangeConflictException)

        {

            m_context.ChangeConflicts.ResolveAll(RefreshMode.KeepCurrentValues);

            m_context.SubmitChanges();

        }

    }

 

Now we can invoke it passing Lambda expression like this:

    [TestMethod()]

    public void UpdateWithAction()

    {

        LinqSampleDataContext context = new LinqSampleDataContext();

        EmployeeAccessor accessor = new EmployeeAccessor();

 

        Employee employee = context.Employees.Single(e => e.EmployeeID == 1);

 

        accessor.Update(employee, t => { t.FirstName = “First”; t.LastName = “Last”; });

    }

 

Unfortunately, the test case can’t pass sometimes. It will throw a NotSupportedException. And the message of exception is:
An attempt has been made to Attach or Add an entity that is not new, perhaps having been loaded from another DataContext. This is not supported.

Why? What happens? The real reason is the entity we handle has some associations with other entities. Please see figure 1:

linqbaseclass01

Figure 1 : The entity with association

If you remove all relationships in Employee table, and re-generated the data model, the test case would pass.

What shall we do? Of course, to remove all relationships of table explicitly is not a good way to solve the issue. It will impact on the whole data model and can’t be accepted when we develop the application. I found one guy, Steve Michelotti, raised one solution to solve it. That was to provide one method called Detach to remove the relationships using partial class:

public partial class Contact

{

    public void Detach()

    {

        foreach (Address address in this.Addresses)

        {

            address.Detach();

        }

    }

}

 

public partial class Address

{

    public void Detach()

    {

        this._AddressType = default(EntityRef<AddressType>);

        this._State = default(EntityRef<State>);

    }

}

 

Good job! But it is not perfect. First, it’s too complex because we must define the Detach method for every entity which has association with others. Second, we can’t abstract it to the base class in this way. In the base class, we don’t know the specific type of TEntity. In this case, we should turn to the reflection technology. It’s my way:

    private void Detach(TEntity entity)

    {

        foreach (FieldInfo fi in entity.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Instance))

        {

            if (fi.FieldType.ToString().Contains(“EntityRef”))

            {

                var value = fi.GetValue(entity);

                if (value != null)

                {

                    fi.SetValue(entity, null);

                }

            }

            if (fi.FieldType.ToString().Contains(“EntitySet”))

            {

                var value = fi.GetValue(entity);

                if (value != null)

                {

                    MethodInfo mi = value.GetType().GetMethod(“Clear”);

                    if (mi != null)

                    {

                        mi.Invoke(value, null);

                    }

 

                    fi.SetValue(entity, value);

                }

            }

        }

    }

 

For EntityRef fields, we may set their values to null by calling the SetValue of FieldInfo class to remove the relationship. However, we can’t do EntitySet in the same way because it is a collection. If set to null, it will throw exception. So I get the method information of the field and invoke the Clear method to clear all items in this collection. Finally, my implementation of Update method is as below:

    public void Update(TEntity originalEntity, Action<TEntity> update, bool hasRelationship)

    {

        InitDataContext();

        try

        {

            if (hasRelationship)

            {

                //Remove the relationship between the entitis

                Detach(originalEntity);

            }

            m_context.GetTable<TEntity>().Attach(originalEntity);

            update(originalEntity);

 

            m_context.SubmitChanges();

        }

        catch (ChangeConflictException)

        {

            m_context.ChangeConflicts.ResolveAll(RefreshMode.KeepCurrentValues);

            m_context.SubmitChanges();

        }

    }

 

Delete operation is similar except we don’t need invoke the second version of Attach (Attach(object entity, bool asModified)). Here is the code snippet:

    public void Delete(TEntity entity, bool hasRelationship)

    {

        InitDataContext();

        try

        {

            if (hasRelationship)

            {

                //Remove the relationship between the entities;

                Detach(entity);

            }

 

            m_context.GetTable<TEntity>().Attach(entity);

            m_context.GetTable<TEntity>().DeleteOnSubmit(entity);

            m_context.SubmitChanges();

        }

        catch (ChangeConflictException)

        {

            m_context.ChangeConflicts.ResolveAll(RefreshMode.KeepCurrentValues);

            m_context.SubmitChanges();

        }

    }

    public void Delete(IList<TEntity> entities, bool hasRelationship)

    {

        InitDataContext();

        try

        {

            if (hasRelationship)

            {

                //Remove the relationship

                foreach (TEntity entity in entities)

                {

                    Detach(entity);

                }

            }

 

            m_context.GetTable<TEntity>().AttachAll(entities);

            m_context.GetTable<TEntity>().DeleteAllOnSubmit(entities);

 

            m_context.SubmitChanges();

        }

        catch (ChangeConflictException)

        {

            m_context.ChangeConflicts.ResolveAll(RefreshMode.KeepCurrentValues);

            m_context.SubmitChanges();

        }

    }

 

Don’t worry about the correctness of final result when we remove the relationship between the entities. The Attach method is just responsible for associating the entity to a new instance of DataContext to track the change. When you submit the changes, DataContext will check the real value in the mapping database and update or delete the record according to the passed entity. Especially, you should take an action such as Cascade in the database if you want to cascade delete between foreign key table and primary key table, see figure 2.

linqbaseclass02

Figure 2: Set the delete rule

If no action, the System.Data.SqlClient.SqlException will be thrown and the message of it when you delete the entity as Primary Key table, like this:
The DELETE statement conflicted with the REFERENCE constraint “FK_Orders_Employees”. The conflict occurred in database “Northwind”, table “dbo.Orders”, column ‘EmployeeID’.

Maybe you notice the InitDataContext method is invoked in all methods to access the data. Its implementation like this:

    private TContext m_context = null;

 

    private TContext CreateContext()

    {

        return Activator.CreateInstance<TContext>() as TContext;

    }

 

    private void InitDataContext()

    {

        m_context = CreateContext();

    }

 

Why do we need create a new instance of DataContext for each method? The reason is Caching policy in DataContext. If you create a new instance of DataContext and query the data from the database though by it, then change its value and execute the next query though by the same instance, the DataContext will return the data stored in the internal cache rather than remapping the row to the table. For more information, please refer to the Linq in Action.

So, the best practice is to create a new instance of DataContext for each operation. Don’t worry about the performance, the DataContext is lightweight resource.

Let’s consider about the concurrency issue. The default option is Optimistic Concurrency. When the values are to be saved, the DataContext would check the previous values to see if they have been changed. If conflict occurs, the DataContext would need to know whether to automatically overwrite the previous changes, keep the previous changes, or somehow merge the changes.

The concurrency issue is not within the scope of this article. We can’t say which one of three ways is the best or worst, it depends on the business situation. Usually, I will handle concurrency by the rule of last submit win. So I encapsulate the SubmitChanges method and define it as visual method, if necessary, the subclass may override it:

public class AccessorBase<TEntity, TContext>

    where TEntity : class, new()

    where TContext : DataContext, new()

{

    private TContext m_context = null;

 

    /// <summary>

    /// It provides the default policy to handle the corrency conflict

    /// </summary>

    /// <param name=”context”>Data Context</param>

    protected virtual void SubmitChanges(TContext context)

    {

        try

        {

            context.SubmitChanges(ConflictMode.ContinueOnConflict);

        }

        catch (ChangeConflictException)

        {

            context.ChangeConflicts.ResolveAll(RefreshMode.KeepCurrentValues);

            context.SubmitChanges();

        }

        catch (Exception ex)

        {

            throw ex;

        }

    }

 

    /// <summary>

    /// Update the entity which was passed

    /// The changedEntity cann’t have the relationship between the entities

    /// </summary>

    /// <param name=”originalEntity”>It must be unchanged entity in another data context</param>       

    /// <param name=”update”>It is Action<T>delegate, it can accept Lambda Expression.</param>

    /// <param name=”hasRelationship”>Has relationship between the entities</param>

    public void Update(TEntity originalEntity, Action<TEntity> update, bool hasRelationship)

    {

        InitDataContext();

 

        try

        {

            if (hasRelationship)

            {

                //Remove the relationship between the entitis

                Detach(originalEntity);

            }

 

            m_context.GetTable<TEntity>().Attach(originalEntity);

 

            update(originalEntity);

 

            SubmitChanges(m_context);

        }

        catch (InvalidCastException ex)

        {

            throw ex;

        }

        catch (NotSupportedException ex)

        {

            throw ex;

        }

        catch (Exception ex)

        {

            throw ex;

        }

    }

}

 

Now, we have a common base class which can be derived to handle the entity. For example:

public class EmployeeAccessor : AccessorBase<Employee, NorthwindDataContext>

{

}

 

You don’t need implement any method. It’s more convenient to access the Employee table by using an instance of EmployeeAccessor:

    [TestMethod()]

    public void UpdateEmployee()

    {

        EmployeeAccessor accessor = new EmployeeAccessor();

        IList<Employee> entities = accessor.Where(e => e.EmployeeID == 1);

 

        if (entities != null && entities.Count > 0)

        {

            entities[0].FirstName = “Bruce”;

            entities[0].LastName = “Zhang”;

 

            accessor.Update(entities[0], true, true);

        }

    }

 

You may let Employee entity derive my base class directly even:

public partial class Employee : AccessorBase<Employee, NorthwindDataContext>

{

}

 

Its behavior is very similar with Rich Domain Model like Martin Fowler said in the article which title is Anemic Domain Model.

 

Download Source Code: LinqSample_Src.zip     368KB

 

Note: The database which my sample use is Northwind in SQL Server 2005 and I add a new column which type is TimeStamp for each table.

Sunday, July 27, 2008

Sprint #2 not Good

In our retrospection meeting on Thursday, I pointed out some issues during the Sprint #2 this month.

1. Sprint Backlog was not exact. Product Owner in our team was not familiar with Scrum and how to analyze the business functions. So we can't do Product Backlog well, I mean, the category and decomposition of backlog items is not good. We didn't find the best way to describe and find out the backlog items we can handle. It was the source of everything couldn't go on well. Without good Product backlog, it was impossible to have the correct Sprint Backlog.

2. The estimate of workload was not exact. In fact, you could never estimate the workload of your project correctly. But you should try your best by using your work experience and skill. If you couldn't make a decision or had no confidence, you should make use of the knowledge of your team members. For example, you should lead your members to estimate the time using planning poker like Henrik Kniberg said in Scrum and XP from the trenches.

3. No effective tracking of task. In Sprint Planning meeting, we assigned the backlog items and tasks to everybody in our team. Then we inputted the information into the Sprint backlog file. But Scrum Master involved in too many trivial issues so that he had no time to track the process of development. The only one way is Daily Meeting. The result is not good because we had no visible policy to represent the status of the task whether it was completed or not.

Something need to changed! Yes, we have to do.

Wednesday, July 23, 2008

TransactionScope Troubleshooting

We always use TransactionScope to handle the transaction based on .Net framework. Before use it, you must add the System.Transaction reference and using it. In my project, we use the LINQ to SQL as ORM framework to access the database. It provides the transaction handling by default that is SubmitChanges() method. If you invoke it, and some errors occur, the datas which were changed will rollback. Besides, we can use the System.Data.Common.DbTransaction to handle the transaction. We can create the instance of it through by invoking the BeginTransaction() method of the connection property of DataContext, then set it to the Transaction property of DataContext. The code sippet like this:

        LinqSampleDataContext context = new LinqSampleDataContext();

        System.Data.Common.DbTransaction trans = null;

        try

        {

            context.Connection.Open();

            trans = context.Connection.BeginTransaction();

            context.Transaction = trans;

 

            context.Employees.InsertOnSubmit(emp);

            context.SubmitChanges();

 

            trans.Commit();

        }

        catch (Exception ex)

        {

            if (trans != null)

            {

                trans.Rollback();

            }

        }

 

However, the ways LINQ provides are only used in the scope of DataContext, if you want to use a couple of DataContext to execute the different operation, you have to use TransactionScope, like this:

        using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew))

        {

            try

            {

 

                for (int i = 0; i < nominees.Count; ++i)

                {

                    Backup newBackup = nominees[i];

                    Ticket ticket = tickets[i];

 

                    //update the information of ticket

                    //mainly add the information of employee;

                    ticket.EmployeeID = newBackup.EmployeeID;

                    ticket.HaveNominated = true;

                    ticket.IsConfirmedByManager = true;

                    ticket.Status = TicketStatus.Enroll.ToString();

 

                    ticketAccessor.Update(ticket);

                }

 

                //update the IsSubmit of backup;

                ChangeSubmitStatue(backup);

 

                //remove the record of nominee in backup table

                Delete(nominees);

            }

            catch (Exception ex)

            {

                ThrowHelper.ThrowBackupException(“Finalizing occurs an error. The transcation will be rollback.”);

                return false;

            }

 

            scope.Complete();

        }

 

Running the code, maybe you will meet an error which is an System.Data.SqlClient.SqlException, and the message of it is “MSDTC on server ‘{Server Name}’ is unavailable.” What happen? It is necessary to make sure whether the Distributed Transaction Coordinator Service is start or not. Please notice I mean you should check your database server instead of the one your software is running. You can open the Administrative Tools and Service, find the Distributed Transaction Coordinator Service and Start it. You’d better set it to be Automatic.

Let’s consider about this situation. The server your software is running is not same machine with your database server. Now running the code, the terrible expection will be thrown maybe this time. The type of it is System.Transactions.TransactionManagerCommunicationException and the message is “Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please enable DTC for network access in the security configuration for MSDTC using the Component Services Administrative tool.”

Please pay attention to the type of exception. It’s a communication error. Although you start the correct service in Database server, but the server your software is running can’t communicate with your database server when we use the distributed transaction. In fact, it’s a security issue. The solution to this problem is set the appropriate MSDTC Security Configuration option on Operation System such as Windows Server 2003 sp1 or Windows XP sp2.

Note: The machine you wanna set should be the one your software is going to run.

To access the MSDTC Security Configuration dialog box follow these steps:
1. Click Start, click Run, and type dcomcnfg to launch the Component Services Management console.
2. Click to expand Component Services, click to expand Computers, right click My Computer, and click Properties.
3. Click the MSDTC tab of the My Computer Properties dialog and click the Security Configuration button to display the Security Configuration dialog box.

What are the correct values? Microsoft give the recommended values refer to the table as below:

Configuration Option Default Value Recommended Value
Network DTC Access Disabled Enabled
Client and Administration    
Allow Remote Clients Disabled Disabled
Allow Remote Administration Disabled Disabled
Transaction Manager Communication    
Allow Inbound Disabled Enabled
Allow Outbound Disabled Enabled
Mutual Authentication Required Enabled Enabled if all remote machines are running Win2K3 SP1 or XP SP2 or higher, and are configured with “Mutual Authentication Required”.
Incoming Caller Authentication Required Disabled Enabled if running MSDTC on cluster.
No Authentication Required Disabled Enabled if remote machines are pre-Windows Server 2003 SP1 or pre- Windows XP SP2.
Enable TIP Disabled Enabled if running the BAM Portal.
Enable XA Transactions Disabled Enabled if communicating with an XA based transactional system such as when communicating with IBM WebSphere MQ using the MQSeries adapter.

You may refer to the figure that I post as below: securityconfig

In fact, it is default value on Windows Server 2003. However, the machine your software is running always is on Windows XP during Unit Test. If you don’t find the key to this problem, maybe you will be confused.

Thursday, July 10, 2008

Select In Sentence with Linq To Sql

Considering the scenario: If I want to look for some records according to the collection of their ID from the data table, how to write the correct LINQ expression? For instance, I create the training plan table in SQL Server 2005. The data structure of this table is as follows:
ID type: bigint
Name type:nvarchar(50)

Notes: the rest columns have been neglected.

Now I have the collection of ID such as IList which contains some items. It will be the in parameter to find the list of TrainingPlan object. If I use the SQL sentence, it will be:

select * from trainingplan where ID in {1, 2, 3, 4}

So how to do to meet the requirement in LINQ? That’s easy, the code snippet like this:

        public IList GetTrainingPlans(IList ids)

        {

            long[] tpIDs = ids.ToArray();

            var query = from tp in context.TrainingPlans

                        where tpIDs.Contains(tp.ID)

                        select tp;

            return query.ToList();

        }

Please note the first line in the method body. Why did I convert ids(IList) to long[] array? When the input params is IList type, and then invoke its Contains() method, It will throw a NotSupportedException. The error information is “Boolean Contains(System.Int64)” has no supported translation to SQL.

But, if it is Array type or List type, it will not. It’s very strange, isn’t it? Yeah, that’s it whichi I am confused. Maybe the big difference is between the class and interface.

Wednesday, July 9, 2008

Cartoon At June 09

At the funeral of the inventor of the hotdog bun.

Tuesday, July 8, 2008

Software War


Here is a very interesting picture. It shows the development, risk and power of Microsoft in other way. Right, Microsoft maybe is an enemy of all software corporations. Everyday, they fight against the Microsoft and try to occupy more market. To invade, or to be invaded, the war between software coporation is completely same as the war between Countries.

How to Write Notice

Scenario: You are an administrative staff of Kabuki Sales Corporation of America's Eastern Regional Office. Your company has been moved to a new office recently. You are asked to send a notice to all customers and partners to inform the new address and phone number.

Example:
Dear Mr. Smith:
As of Monday, July 1, 2008, Kabuki Sales Corporation of America's Eastern Regional Office will be located in our new offices and warehouse building at 401 Grandiosa Boulevard, Tampa, Florida, 33715. The telephone number for this new location is 550-5428.

Our Manufacturing Division will remain at 2550 Santa Fe Avenue, in St. Petersburg.

I have enclosed our most recent brochure on robotic equipment for your review. I hope you find it interesting.

Your Sincerely
Kevin Johnson

Monday, July 7, 2008

How to Write Announcement

Scenario: Your company will arrange a two-day Sales Conference. All the sales staff will be invited to the meeting. You are asked to send an email to announce the place, time, arrangement and detailed meeting agenda.

Example:
This year's sales conference will take place on Thursday, 12 September at the Tudor Gate Hotel, 1 Princes Way, London. All sales staff from regional offices will attend.

Staff will be booked into the Tudor Gate Hotel for two nights, 11 and 12 September, and will return home on the morning of 13 September. All meals will be provided at the hotel from dinner on Wednesday evening to breakfast on Friday morning. On arrival, staff should collect their identity badges and conference folders from the special reception desk in the hotel foyer.

It is expected that the conference timetable will be as follows:
9.00 Sales Manager's Opening Speech
9.30 New Product Information
10.30 Coffee Break
10.45 New Product Information
12.30 Lunch
2.00 Regional Sales Reports
3.30 Tea
3.45 Regional Sales Reports
5.30 End of Conference

Conference Committee

Cartoon At June 07

The cannibal toys are horrible. Please keep a distance from these.

Sunday, July 6, 2008

Cartoon At July 06

Oh, no, she didn't tell me this is a chick flick.

chick flick: (also "chick's flick") is slang for the film designed to appeal to a female target audience. The term was first used in the 1980s, a decade during which such chick flicks as Beaches were released. Although many types of films may be directed toward the female gender, "chick flick" is typically used only in reference to the film with heavy romantic themes.

Saturday, July 5, 2008

Travel Aroud LiYe

Liye is an ancient town in the southwest of Hunan province and nearby Chongqing. It is famous for culture relic from Qin dynasty. The culture relic is called "Qin Jian". If you are familiar with the Chinese history and traditional culture, maybe you know Qin dynasty was a powerful country in 221 B.C. when the first emperor of Qin unified China. "Jian" means letter or article or diary, it was written by knife on the bamboo paper. That's an earlier implement for writing in China. "Qin Jian" recorded the great amount of historic materials to help us research the Qin dynasty history.

In Apr 11, my wife and I visited Liye with my colleagues. At first we went to the Xiushan which is a small city in ChongQing by train, then arrived at Liye by bus in the evening. It was not dark. After supper dinner, we couldn't wait to visit the ancient relics of "Qin Jian". However, it had became the ruins. Nothing was left because all "Qin Jian" were collected by government and were moved into the museum to protect properly. To attract more visitors, the local government was going to build the museum. But now it was only plan.

We were all depress. Fortunately, the town's buildings were decorated in the traditional style and showed the beautiful scenery. We walked through the streets in the night, experienced the feeling of stranger suddenly. In fact the scenery here was nearly similar as my hometown.

Next day, we went to a small town called "Xichehe", the English meaning is the river where some guys washed the wagons. As tourist guide said, the origin of the name was from a legend. It said that one man was back to hometown through by Xichehe driving a wagon. When he arrived here, the horse lose the man's control suddenly and rushed into the river so that the wagon was fell into the river. After the horse drank the water of the river, it was back to normal and pulled the wagon following its master's order again.

Maybe the legend painted some mysterious colors for us, but this small town is really old, broken and dirty. We were all disappointment especially me, because the scenery here was same as my home country several decades ago.

It is the area where the minorities located around Liye such as Miao, Tujia. So the custom of their life and the style of their building are different from ours. In that afternoon, we went to the village which is belong to the indigenous citizen of Tujia minority. They were all warm and hospitable. They took their wine produced by themselves and invited us to their home. They are all good at dancing and singing. After lunch dinner, they performed many exciting dance for us. It was great.

The travel time was too short, we had to be back. Although the scenery there is not beautiful, but a lot of thing during this travel gave me deep impression still.

Cartoon At July 05

"Oh, Dave, you have GOT to see these adorable alien costumes! Talk about realistic!"

adorable:
1. very attractive or delightful; charming;
What an adorable hat!

2. worthy of being adored.

Work Time in Extreme Programming

InfoQ posted the news about Sustainable Pace in XP on May 2, 2008 which title is Does Sustainable pace mean a 40 hour week? That's an interesting topic about agile development. The article focuse on the correlation between the number of work hours weekly and sustainable pace.

As Henrik said, every programmer has a "Maximum Productivity" which depends on his talents. It called "Current Capacity" which is the highest amout of work which can be done whithout lowerring the quality. So he suggest we shoud make sure the every developers' current capacity at the beginning of the project development, otherwise keeping the work week at 40 hours sounds like waste. He said:
If we can hold sustainable pace of say 45 hours per week, as opposed to 40, we gain more than one month of extra development time during a year! Especially in start-ups, such as where I work, 40 hours a week is a luxury one can't afford.

But other guys disagree with Henrik's opinion. They suggested that increasing the number of work hours does not linearly increase the productivity. If we work overtime, teams tend to more destructive than productive. When working long time, the productivity begins to decline.

So the key point is how to value the degree between the number of work hours and productivity. We cannot increase the number of work hours to improve our business value. We must consider about the ability of undertaking the pressure.

The summary of this article is that the important factor to sustain pace is not number of hours worked, but to think creatively and accomplish more in less time. The sustainable pace revolves around a lot of factors like work culture, technical debt, motivation etc. It might have a small influence from number of hours worked, but that is not the best metric to start with.

English Conundrum

I

What ship is found in a church?
Worship.
What ship needs a group of worker?
Membership.
What ship does everyone enjoy?
Friendship.

II

It hangs stiff and hard,
Under a man's cloths,
Waiting to fit a familiar hole.

Guess what? The key is a correct answer to it!

When You Are Old

William Butler Yeats(1865 - 1939), Nobel Prize winning Irish dramatist, author and poet wrote The Celtic Twilight(1893).

When you are old and gray and full of sleep,
And nodding by the fire, take down this book,
And slowly read, and dream of the soft look
Your eyes had once, and of their shadows deep;

How many loved your moments of glad grace,
And loved your beauty with love false or true,
But one man loved the pilgrim Soul in you,
And loved the sorrows of your changing face;

And bending down besides the growing bars,
Murmur, a little sadly, how Love fled
And paced upon the mountains overhead
And hid his face amid a crowd of stars.

Friday, July 4, 2008

Cartoon At Jul 04

Thanks to the amazing new Zap-a-Slacker, parents are able to send a mild electrical current to their daydreamy Little Leaguers.

slacker: a person who shirks their work or duty (especially one who tries to evade military service in wartime)

daydreamy: please refer to daydream.

What Makes a Good Stand Up Meeting?

In Scrum, Stand up meeting is called daily scrum. It's very important to prompt the team members' communication, and helps Scrum Master know about the process of development. As its name implies, the meeting should be hold daily and ask everybody to stand.

In normal, during the daily scrum, each team member should provide the answers to the following three questions:
1. What did you do yesterday?
2. What will you do today?
3. Are there any impediments in your way?

The daily scrum meeting is a statue check where the team meets and updates each other about what's going on. It provides the daily focus on the work being done:
1. Same time and place
2. Avoids overhead of finding a place daily
3. Avoids overhead of team trying to figure out where and when is today's meeting
4. Let's chicken's know where and when
5. No more than 30 minutes
6. Scrum Master asks pigs three questions
7. Scrum Master is responsible for making the decisions
8. Scrum Master is responsible for noting and solving work impediments
9. All discussion other than replies to 3 questions deferred to later meeting

Though it is very short, but very important. So scrum master should try one's best making the team's daily scrum good. Marcie Jones give some tips about Daily Scrum Meeting:
1. Don't be late yourself.
2. The meeting needs to start on time.
3. If #1 and #2 really can't happen, experiment with different meeting times.
4. Cut off inappropriate discussions.
5. If the updates are too vague, ask question.
6. Ask your team permission to keep trying it daily for say, one more iteration while you try to fix these problems.

Do you agree with these opinions? Do these ways really help scrum master improve the quality of daily scrum meeting? Please try it!

Two Poems of Shelley

Percy Bysshe Shelley(August 4, 1742 - July 8, 1822) pronounced was one of the major English Romatic poets and is widely considered to be among the finest lyric poets of the English Language. He is perhaps most famous for such anthology pieces as Ozymandias, Ode to the West Wind, To a Skylark, and The Masque of Anarchy. However, his major works were long visionary poems including Alastor, Adonais, The Revolt of Islam, Prometheus Unbound and the unfinished The Triumph of Life.

I

A widow bird sate mourning for her love
Upon a wintry bough;
The frozen wind crept on above,
The freezing stream below.

There was no leaf upon the forest bare,
No flower upon the ground,
And little motion in the air
Except the mill-wheel's sound.

This poem is the real image of our victims in Earthquake. Bless them!

II

Music, when soft voices die
Vibrates in the memory—
Odours, when sweet violets sicken,
Live within the sense they quicken.

Rose leaves, when the rose is dead,
Are heaped for the beloved's bed;
And so thy thoughts, when thou art gone,
Love itself shall slumber on.

Shelley is one of English poets whom I like. He is so popular in China mainly due to his great poem "Ode to the West Wind". Especially the last sentence of this poem is most famous:
If Winter comes, can Spring be far behind?

It encourages many people in difficulties to struggle for their dream, their ideal.

Encapsulation of Change

The change of the requirement is an enemy we must fight against when we develop the software project. Sometimes, the change of the requirement is endless, so we have to delay our project. Of course, our customers have not been patient with our delay. They would rather believe the difficulty we meet because of the change of requirement is the foxy excuse. The change is like a sword of Damocles over our head. As brooks said, there is no silver bullet in the process of software engineering. It is same that we can't find the "silver bullet" to solve the problem of change. It's a mission impossible. But don't scare of it, we must face its difficulty actively. So Kent Beck, the leader of XP (Extreme Programming), announced that we must embrace the change. He provided the solution to solve it from the angle of Software Engineering Methodology. And now, this article will address issue how to solve the change in the process of developing the software project from the angle of Software Design Methodology. That is "Encapsulation of Change".

Design Patterns is the best interpretation of "Encapsulation of Change". No matter we use the Creational Pattern, Structural Pattern, or Behavioral Pattern, our objective is finding the change in Software, then encapsulating the change with abstraction and polymorphism. So, at the beginning of design, we not only implement the User Case, but also mark the existed change. Encapsulation of change means to discover the change or find the change. It is very important.

The objective of Creational Pattern is to encapsulate the change when creating an object. For example, it builds the specific abstract factory class when we use Factory Method Pattern or Abstract Factory Pattern. So this factory class encapsulates the creational change in the future. Bridge Pattern encapsulates the change when creating the object inside the product, so that we can replace it with the new object to meet the change of requirement.

As far as Structural Pattern, it focuses on the composite style between the object. When we change the structure of object, in fact the dependency between the objects would be changed. Of course if you want to use the Structural Pattern, you will use not only encapsulation, but also inheritance and aggregation. For example, Decorate Pattern describes the dependency between the decorator and the decorated object. The decorator is the abstract class so we can handle the change of the structure of the object. Similarly, Builder Pattern encapsulates the dependency of an implementation of the object. Composite Pattern encapsulates the recursive relationship between the objects.

Behavioral Pattern mainly focuses on the behavior of the object. It abstracts the behavior of the object into the specific abstract class or interface, so that it becomes more extensible, because the behavior is the most unstable part in the architecture. For instance, Strategy Pattern abstracts the strategy or algorithm into a sole class or interface in order to encapsulate the strategy. Command Pattern encapsulates the request. State Pattern encapsulates the state. Visitor Pattern encapsulates the style of visiting. Iterator Pattern encapsulates the algorithm of iteration.
By using the Design Patterns and encapsulating the change, it ensures the extensible of the architecture. Maybe we can’t sweep the nightmare which was brought by the change, however, if we can foresee some changes at the beginning of the design, it will avoid the trouble caused by the change in the future. So we can say Design Patterns should be a nice spear to fight against the change, although we can’t find out the “Silver Bullet”.

Joy & Disappointment

Zheng Jie lose the match with Serena Williams at Wimbledon unfortunately. Yes, I am very disappointment, however, the joy is more, because She have created the history especially she is an unseeded player. Zheng Jie's performance in Wimbledon is great. She is our pride absolutely.

Before Wimbledon 2008 opened, who knew the player who required a wild card from Wimbledon organizers can sprint into the semifinal? In semifinal, she lost the match, but win the respect of the world.

Zheng Jie is the first Chinese woman to play in a Grand Slam semifinal in singles. I believe she will try to keep going next time. Thank Zheng Jie for your effort and excellent performance. You are our hero. See you! I expect you can create the surprise again.

Thursday, July 3, 2008

Agile is Not Easy

On Apr 30, InfoQ posted an article "实践敏捷很容易:为期两天的“迷你”敏捷项目". It introduced the Code Jam that was hold by ThoughtWorks. Absolutely, this form of Code Jam was very useful and interesting. It would help the developers to master the agile methodology, idea and principles. But I don't be agree with the author's opinion partly. I think whether agile is easy should depend on the specific situation.

In Thoughtworks, most of the developers are familiar with Agile Methodology. They are all agree with the idea and principle of Agile. So we don't need persuade them to accept this methodology. They know about the process of agile development. They are used to using TDD and Pair Programming to develop the software. They enjoy the communication. They appreciate for the efficiency of Standup Meeting. They are fond of Retrospection to summarize the good experience and shortage to help improving the development process. In a whole, they are excellent team members and the team is more professional. This team is a real agile team. In this team, maybe everyting is going on well.

But the thing is not alway nice. If your team members all are fresh guy with Agile Methodology, what should you want to do? More serious if most of team members have not master the basic skill for developing project, what should you want to do again? Can you say: Agile is easy?

In this situation, How to implement agile and what to practice agile are the first task you should complete if you are an agile coach or ScrumMaster in Scrum. You should introduce some issues related Agile Methodology to everybody. You should let them master the agile knowledge, and think everything during developing project in agile thinking. You should be careful everytime and pay attention to every cicle of software development. If some mistakes happen, you must correct them in time. You should negotiate with your leader and customer, and try your best to prevent your members from interfering by others. It's your responsibility as "Collie". At the same time, you are a guide to lead them walking on the right way to destination. Besides, you should build the harmonious environment to prompt the communication between the members.

There are many problems you should solve as well. Anyway you can't escape. Unfortunately, my project is in this situation now.

Cartoon At July 02

With three hours in costume still ahead of them, Mike was becoming increasingly dismayed as Ed wolfed down his third bowl of beans and cabbage.

dismay
[Webster 1913] To disable with alarm or apprehensions; to depress the spirits or courage of; to deprive or firmness and energy through fear;

Be not afraid, neither be thou dismayed. --Josh
What words be these? What fear do you dismay? -- Fairfax

[Webster 1913] To render lifeless; to subdue; to disquiet.

Do not dismay yourself for this. --Spenser

[Webster 1913] Loss of courage and firmness through fear; overwhelming and disabling terror; a sinking of the spirits; consternation.

I ... can not think of such a battle without dismay. --Macaulay

wolf down
eat hastily;

The teanager wolfed down the pizza.

Nobody

Nobody is willing to be nobody.
Fame, Money, Beauty, is your dream?
Be Ordinary, Poor, Humble,
It’s a nightmare, isn’t it?
Anyway, only somebody can be somebody
That’s the truth you can’t change.
Somebody said, The world is flat,
All men are created equal, however
Not be born equal!
In fact, It is absolute truth.
So you must fight!
Because, the Fate is your Enemy.

Scalability Principles

One of the most important aspects which should be considered about during you design the architecture of system is scalability. In the future, the requirement will be changed, the system will overload, the amount of user will be increased. If you don’t consider about the scalability of software system, you system is becoming rubbish.

Several days ago, InfoQ posted the useful article wrote by Simon, it was Scalability Principles. This article presented some principles and guidelines for building scalable software systems.

To build scalable software systems, you should try your best to decrease processing time. Thus the system can handle more user requests in the same amount of time. The article gave some strategies to achieve this goal.
* Collocation: reduce any overheads associated with fetching data required for a piece of work, by collocating the data and the code.
* Caching: if the data and the code can’t be collocated, cache the data to reduce the overhead of fetching it over and over again.
* Pooling: reduce the overhead associated with using expensive resources by pooling them.
* Parallelization: decrease the time taken to complete a unit of work by decomposing the problem and parallelizing the individual steps.
* Partitioning: concentrate related processing as close together as possible, by partitioning the code and collocating related partitions.
* Remoting: reduce the amount of time spent accessing remote services by, for example, making the interfaces more coarse-grained. Please consider the first law of distributed computing - don’t distribute your objects.

Scalability is inherently about concurrency. As Martin Fowler said in Patterns of Enterprise Application Architecture, concurrency is one of the most tricky aspects of software development. Whenever you have multiple processes or threads manipulating the same data, you run into concurrency problems. To handle the concurrency issue is very difficult, you have to consider many aspects, e.g. lock, resource contention, deadlock, concurrent transaction, etc. This article presented some simple principles that can help when building scalable systems.
* If you do need to hold locks (e.g. local objects, database objects, etc), try to hold them for as little time as possible.
* Try to minimize contention of shared resources and try to take any contention off of the critical processing path (e.g. by scheduling work asynchronously).
* Any design for concurrency needs to be done up-front, so that it’s well understood which resources can be shared safely and where potential scalability bottleneck will be.

In order to build a successful software system, you need to know what your goals and what you’re aiming for. Your design must meet not only the functional requirements, but also non-functional ones. Non-functional requirements include performance, security, scalability, interoperability etc. Before you build the software system, you need to know related information about non-functional requirements as early as possible.

You must keep test continuously in order to satisfy the non-functional qualities of a system.

Probably the most important principle for building scalable system is that, if you need your system to exhibit this characteristic, you have to design it in up front. Even you fulfill RUP or Agile methodologies, it is necessary still. At least, you should outline the overall view of the system architecture.