Thursday, July 14, 2016

Chrome Dev Tools - Console Tricks

I’m old enough to remember debugging javascripts using alert statements littered all over the code base. However Firebug from Firefox changed the game. Nowadays I use Chrome developer tools.

Let me quickly share some less known tricks around the Chrome developer tools’ console.

1. console.log(‘message’) - This is straightforward log of a message string, nothing special.

2. How about c like string formatting of the log statement?
console.log(‘Number %d, a string %s an object %O went to a bar”, 123, “message”, { country: “Sri Lanka” })

3. Another useful way to log objects specially if it's a collection (array) is to use console.table(). Instead of having to expand and collapse individual objects to check them, you can see all of them directly in tabular form.

4. Different log levels. Just like any logging framework console has different levels of logging in addition to the console.log called info, warn and error
console.error includes a stack trace as well

5. How about printing the stack strace? This is specially useful when you want to track where an error/special condition originates from.

console.trace(“message”) will print a whole stack trace to the point at which console.trace is called.

6. Grouping logs together
It’s also possible to group a set of log statements together so you can collapse expand them and also distinguish this group from other statements.
Here’s how above works"Group starts");
console.log("Testing 1");
console.log("Testing 2");
console.log("Testing 3");

I found this to be really useful when logging inside iterative blocks.

7. Similar to group we can use console.time() and console.timeEnd() to log the time taken to execute a set of js statements

8. Again similar to above we can use console.profile() and console.profileEnd() to add a CPU profile for further investigation. It can be seen under the ‘Profiles’ tab in dev tools

Sunday, May 29, 2016

First Impressions at Work - Employers Point of View


As a consultant, I have been moving between jobs more regularly than a normal employee would. In the past I have moved between couple of jobs as an employee too. Along the way I have picked up a few tips regarding how to make the life of a new recruit better. Sadly, as excited you are on taking a new job, there’s always the inevitable disappointments associated during the first few days at the job. What I want to discuss is, as an employee how you may improve the situation. This is quite important, as our human nature is to put more weight towards how you feel at the first few days at the job. Unfortunately these first impressions might go a long way towards how you feel about the job overall.

So it makes sense from the employer's’ point of view to at least get the first few days correct. Try to welcome the new team member with enough professionalism and friendliness which will re-enforces that he has made the right decision.

I will start with the real fundamentals and list down some of the more exciting ones towards the end. (Some of the items may obviously biased towards software engineering)


  1. Please make sure they actually have valid access to the building. I have worked in places where for few weeks my building access was not sorted. You basically get a visitor pass and it’s not fun to ask for a colleague to escort you to the toilet every time because you may be locked out of the working floors if you walk out.
  2. Have their desk ready. No - you don’t need flowers and balloons, just a clean desk with a computer is fine. Yes, that includes a chair too.
  3. Having a desk and computer is a good start. But also make sure they can actually login to the network and have proper access rights. I have worked in big corporations and government organizations where it takes them few weeks to sort out Active Directory access.
  4. Please take the time to show the new guy the office facilities and how to use them. Toilets, printers, kitchen, vending machines...anything. Teams may overlook these simple things specially if the team is quite busy, but a few minutes of effort may go a long way in communicating that the team actually cares about the new employee.

  1. The computer should have all the necessary software installed. If not installed, the installation process should at least be properly documented (in a wiki page with links to each software package). This can be the difference between someone either spending days to set their machine up or realising after few weeks that they’ve set it up in a slightly different way so they have to start it over again.
  2. Should be given access to credentials required to login to different systems. If installation of security certificates or keys are required, this is the time to do so. It will certainly help to have a password/key management software like keepass or lastpass.
  3. Admin rights to the computer. I can’t stress this enough - especially if you are a software engineer. Not having admin rights to the box could easily add up to days and days of support calls and productivity issues down the line. Not to mention the sheer frustration of the user.
  4. May be have a starter pack documentation in wiki.
    1. Page with the team with photos and details
    2. Organization/Project terminology
    3. FAQ
  5. Organize a meeting with few team members they would work with and clearly provide details around what they will be working on and the expectations. This is not to scare the poor fellow, but to show that you have planned his work and is serious about the responsibility of each team member.
  6. Get an experienced team member to briefly demo some of the common workflows and tools used. It could be JIRA, IDEs, Version Control, Build Systems, Messaging etc…


Even if you get most items from the above two lists,  it’s imperative that the new recruit feels part of the team.
  1. Take them around the office floor and introduce to each team member - one by one. Doesn’t matter if he doesn’t remember any single one of them - it’s the act that matters. This should also cover off-shore team members (or clients) if that makes sense.
  2. Invite them for lunch. A new recruit is reluctant to suddenly ask strangers to join in for lunch. It’s not a great feeling to have lunch on your own at your first day.
  3. Invite them for coffee - specially if lunch doesn’t work out.
  4. Get a friendly, outgoing (but knowledgeable) team member to act as his buddy for few weeks. This can be set up informally before the arrival so that it doesn’t become too uncomfortable.

There are some additional actions you can take which can really make an impression.
  1. Organize a team lunch on behalf of the new recruit. This could very well be the regular weekly lunch the team goes out to - but it has a theme just for 1 week. To welcome the new recruit. If the team culture is different, you may replace lunch with drinks after work.
  2. The lunch could be sponsored by the company. This gives the existing team members a tangible benefit due to the arrival of the new recruit.
  3. Offer to sponsor a tool or a software that the new recruit would like to use for his own efficiency. This really drives homes the fact that the company cares about the time of the new recruit - not just his output.
  4. Encourage them to improve their skills and offer to sponsor for online courses that they are interested about. Important thing here is not to be restricted to purely work related courses. Could be anything. (Coursera offers good quality courses from reputable universities for affordable pricing)

Monday, November 02, 2015

The illusive mission statement

Can you remember the mission statement of your place of work? If you do, then you truly are a special one. When I started my career, mission statements stood out like this carefully stitched up collection of words that emits its grandeur nature to all around us. It didn’t take too much time for that grandeur mission statements to pale into useless set of empty words that doesn’t convey anything objective to anyone in the organization. Except of course for the for handful of people who actually come up with it, because that’s the sort of things they get paid for.

But how do you measure the value of such inspirational one liners? Can they be useful in lining up a team towards a certain goal? It’s well known fact that when you know what you are working towards, chances of you actually getting there significantly increase. So how do you as a leader of a team (Could be your team at work, your family or your sons’ football team) assist your teammates to collectively work towards the same goal, by getting these one liners into their subconscious?

In his book ‘Essentialism’ Greg Mckeown nicely put these kind of statements into a simple diagram of 4 quarters.

The first quarter is not worth discussing about. 

The mission statement you see in 99% of big enterprise falls into the second quarter of ‘Inspirational’ but way too vague to be of any use to anyone. 

Your day to day goals and KPIs could well fall into the fourth quarter where they are very very clear and concrete, but mundane and doesn’t carry a lot of inspiration. True - they have their place, but it’s not when you want to inspire a bunch of people towards a goal.

The third quarter is where the magic is. It’s where you get inspirational yet concrete enough goals. These inspire people as well as giving them a sense of how their everyday tasks can relate to it. One example Mckeown comes up in his book is a mission statement by a government agency in UK.  In a  project to improve the government IT initiative, after the lead of the project came up with a  mission statement that was both inspirational yet resonated with both the project team as well as the citizens in general. It was  “To Get everyone in UK online by 2017”. Clear, concrete yet still inspirational. These sort of statements can be an essential tool for project people to gauge their everyday tasks against the end goal of the project.

Now extending this idea to the software development world, I think any project substantial enough can actually benefit from having such a clear, concise yet inspirational statement. You don’t have to call it a mission statement. But it should be communicated to team members early and clearly, and may be repeatedly. There’s always the risk of ending up with a watered down statement which ends up in the second quarter - which can actually do more harm than having none at all. But it may well be worth the risk, especially if you know the pitfalls as we do now.

When you are stuck with making tough decision that doesn’t have the necessary data behind it to make a highly objective decision, the statement could be your guiding light. I remember once a customer (CEO of a small business having around 20 staff) telling me that the goal of the project was to get rid of 2-3 staff members each year. Not that I call it a truly inspirational goal (specially if you are one of his staff members), but for a shareholder or the external consultant like I was - it could well be. It gave me clear idea about what this customer is after and the feeling that this project could really make a big difference to this company. Those are the kind of feelings and thoughts that needs to be nurtured so that they fuel your team during the tough times.

Saturday, September 05, 2015

Making the best use of Architects in an Enterprise

I have worked with many types of Architects over the past 10 years. Most of my work experience is in the Enterprise world, which gave me a chance to work with more Architects than a non Enterprise developer would get. They come in different shapes and forms, Software Architects, Product Architects, Technical Architects, Solution Architects and drums roll….Enterprise Architects. Although you may not see this in formal job adverts there’s such a thing as Architect Astronauts as well.


Although we got architects coming in different shapes and sizes, unfortunately I have not seen many adding a lot of value to a project delivery. Yes, some of the above mentioned architects by definition of their role does not have anything to do with a delivery, at least not in a direct way. I won’t be stretching the truth if I say that architects are people who have been in the industry for a considerable amount of time and have done a lot of delivery work themselves - albeit prior to them being architects. Given that, it’s counter intuitive in a way, that Enterprises in general values architect input in more strategic and high level areas and ignore the value they could provide in shipping software.

First, I’d like to discuss whether it’s beneficial for an organization to encourage its architects to play a more involved role with the delivery teams. Then in the cases where it is, I’d like to discuss some techniques one can use to get the best out of the role ‘Architect’. During the process the I’ll also try and see how an architect him/her self could benefit by being more involved in project delivery.

Benefits of Architects involving in Delivery
In my past experience most architects engage with the delivery teams either at the start during elaboration or at the end during roll out (specially when things turn out to be nasty). In my opinion this is not the optimum use of an architect. Software Architecture by definition covers the aspects of Software Development that’s hard and costly to change. By dis-engaging with the delivery team after the initial phases you are losing valuable feedback that you get during the lifetime of project development. Feedback that is essential in identifying the pieces of your software that need to change, adapt to the ongoing changes and new findings. Most of the time architect is not to be found until the project gets close to delivery and at that point most of the decisions that have been made have become too difficult to change.

Another disadvantage of not being in touch with the teams is that the architect losses upto date information re: how the technology landscape is moving. How can a person who is suppose to make organizational level technological decision get himself disconnected with the best possible source of relevant technological information. A dis-engaged architect has no chance of keeping the product architecture respond to changing requirements while making sure it also aligns with the enterprise architecture vision as well.

So what would be the best possible way of getting the best out of the Architects. Assigning him delivery tickets from your sprints might not be the best possible tactic. Yes, there’s nothing like delivering cold hard code that gets you right in to it. But considering the varied aspects of an Architects’ role, burdening him with direct delivery artifacts might not be the most efficient use of his time. It might even be detrimental to the team. The team velocity might see weird up and down spikes based on how busy the Architect is with other commitments.  

Tactics for enhancing Architect Engagement


Daily Stand Ups
Daily stand up is an efficient method of getting to know the pulse of the team. Most importantly it would give the architect an opening to know the challenges the team is facing. These challenges / problems could be the spark for more significant architectural changes. Alternatively architect could intervene early to direct the team in case the problems are due to a mis-understanding of the requirement or technology. By assisting the team through these difficult tasks the Architect could also gain some trust and appreciation from the team members which is vital in the long run.

Code Review (of Architecturally significant pieces)
In general code reviews is a great way to intervene and understand what’s actually delivered by the team. But considering the various commitments an Architect might have, it’s important that only a selected set of code reviews are performed by the Architect. I think the selection should be driven by the Architect, to examine architecturally significant deliveries. An example is any framework code done in the early phases of the project. In-fact I’d prefer the Architect to write or at least heavily get involved in writing most of the framework code, let alone reviewing it. In addition to framework bits, some other significant areas could be integrations, complex business logic, high usage areas etc…It’s also important to let go certain areas of the code both for efficiency reasons and also to enhance the decision making and ownership of the team.

Pair programming
Similar to code review, pair programming is a great way for the architect to keep a close eye into any architectural feedback coming from the team. It also helps the team members to learn from a more experienced engineer. Even little things like development utilities and shortcut keys could assist the development of the junior team member. It can work the other way too, specially for a time poor architect, this is a good way to learn a few tricks of the trade from a developer.

In a large enough project, it’s inevitable that you need to some research and proof of concept work before you commit to a particular technology or method. An example would be to check which logging framework to use for the logging in your project, specially if there are specific requirements around logging in the project. Architects with their wider understanding of the projects’ landscape put together with their experience with different technologies and processes are ideal candidates to carry out these pieces of work commonly known as Spikes.

Some of the key technological and tool selections generally need to be backed up by a solid spike. As with everything else, this benefits both the project and the architect, not to mention the team members.
Tooling Research & Purchasing
I am firm believer of usage of proper tooling in any given software development project. Sometimes its unfortunate that some of our Enterprise clients don’t put enough emphasis on the tools, which can end up costing them thousands of dollars of programmer time and rework. One thing I’ve noticed is that when the source of a tool request is a developer, the chance of approval diminishes. But a knowledgeable architect can do these things and own a tool request or an upgrade. Not only an architect can really justify the availability of a tool among team members, he/she is in a much better position to justify the purchase of a tool to a customer as well.

By being engaged in this kind of productive role towards the team, makes the Architect a popular figure among the team - one of our own -, a key requirement for a healthy working relationship between the development team and higher ranks in the company.

Process Improvements
Since Architects are not fully engaged in the day to day activities of a project, once it goes into full swing, he/she is in a great position to provide valuable feedback re: the software development process. For one this architect should be able to convert daily practices into more measurable heuristics on which decisions can be taken. A good understanding of current trends in software development practices as well as hands on experience on delivering software are key to understanding the (sometimes implicit) process quirks within a team and modelling them to known patterns.

This modelling is quite important, as that disconnects the actions of individuals or team, from the individuals themselves. When providing feedback to somewhat intangible aspects like ‘process’, the experience of architects can be drawn in to model the day to day activities into known software development process models.

In an Enterprise environments, software architects in general play a more strategic and advisory role. While appreciating the value of that, it’s imperative that software development teams also utilize architects to improve their capabilities around delivering working software.

An important shift in thinking about this is to engage the architect in one of many contributory roles. Not only these tasks will help the team to deliver software, it will help the team to understand the overall picture external to their delivery, will also help junior members to look up to senior members of the community and improve their skills. On the other hand it will also assist the architects themselves to get in touch with what’s happening in the industry at ground level and valuable project level feedback that could then be sent up in to the organization hierarchy.

Do your architects play a role in delivering software in your organisation? If so, what techniques do you use?

Wednesday, July 08, 2015

Integration Testing - Real World Challenges

Testing is an integral part of any software development project. While everyone understands and acknowledges the importance of Unit testing, Integration testing is also equally important in an Enterprise software development project. Integration testing is the kind of testing where multiple parts/modules of a software system is tested as a group. The multiple modules could be different layers of the system including the database or it could be the application together with external systems.

Enterprise applications often have lots of integration with external systems, systems that are often out of your control. These external systems may not be consistent when it comes to availability, performance or state, posing challenges to day to day running of an integration test suite, continuous integration/deployment set ups or maintenance of an integration test suite.

Whether it’s integration testing covering different parts of your own application or covering external systems, integration tests possess different challenges that is not seen in Unit testing.

I’d like to share with you some important learnings around testing in a large Enterprise application development project.

It’s not just a ‘test’

Testing is not all about making sure your application is in good order. In a big enterprise applications, tests are a good mechanism of understanding how different parts of your system work. It’s essentially a type of documentation for developers while they are developing it. Further it’s a great tool for developers and support personnel during the maintenance phase. This is especially important considering the likelihood of key personnel of the project moving on with time.

In addition to being a great source of documentation, the tests also act as a great design reviewing tool. If you ever struggle to write a test for a particular component, it’s a very reliable source of code smell. Give a moment to rethink the structure of your code and components to see whether it can be tested easily. This is especially true for Unit tests.

Continuous Integration / Build System

A major benefit of a continuous integration system is quick feedback on the state of your application so that developers can quickly fix whatever is broken. However with the increasing number of tests, the build will start taking a longer and longer time to complete. This is increasingly true for integration tests as in general they took more time. So although having more and more tests is generally accepted as good practice, there may be a point of time when the time taken to run all of these tests could be having a negative impact on the project.

Also the dependency of integration tests on external systems mean that your build will break for issues that is outside the scope of your project.

Considering all of these, after a certain point, it no longer becomes feasible to run the full suite of integration builds for each check-in.

Instead, a more pragmatic approach is to have 2 or multiple level of Builds. A low cost build comprising of all Unit Tests and possibly a selected subset of integration tests can be run for each check-in. The full test suite can be run either in the middle of the day or overnight just to give the team some sort of an indication on the overall state of the application. This strategy will still encourage the team to write more tests while keeping the productivity in check.

Also it would be useful to differentiate integration tests that depend on external systems vs integration tests that just talk to internal systems or databases. This could be useful when troubleshooting test failures. You don’t want the team to be consuming time to figure out what’s wrong with an integration test to later realise that it’s due to an external system not playing nicely. It should be plain obvious by just looking at the test name or the failure whether the issue is due to an external system or not.

Seeding Data

Another contentious topic regarding integration tests is how the whole setup of data works. In the real application any operation will require the application to be in a certain state. The seeding of data is intended to bring the application to this state by artificially pushing data into the database or in memory data structures.

One important aspect of seeding is that after every test method the application should be back to its original state. In the context of database integration tests this would mean seeding data at the top of the test and then deleting it at the end. This act could become quite costly quite soon with deep object hierarchies and complex relationships among entities. Also it could be quite a lot of work just to set up the seeds compared to the actual code that does the testing. In fact if you are spending a lot of time in your tests writing code that seeds data it could be an indication that you need to rethink the tests or your application code.

After all these challenges posed by integration tests, one can not argue the importance of an integration test suite in an enterprise application. However with careful strategies we can overcome these challenges.

  1. Write integration tests to check end-to-end traversal of your business process. Not the business logic itself
  2. Make integration tests faster by using in memory databases like Sql Lite
  3. After a certain point, stop running the full suite of tests for each check-in. Opt for a multi level build system where integration tests are run periodically independent of check ins.
  4. Categorize the integration tests so that it gets easier to identify root cause when they fail

Friday, February 13, 2015

Grass is green in Brownfield as well

After a lengthy stint of ‘Brownfield’ development I’m moving on to a ‘Greenfield’ project. I thought it’s time for a bit of retrospection.

Contrary to many other fields, the software field tend to look down on artifacts created by others and especially prior to the current times. Terms like ‘legacy’, ‘brownfield’ reflects this fact very well. In Literature people still hold classics such as ‘Shakespeare’ or even more contemporary works to great heights - they discuss these works and let them be studied by newcomers to the field. If you try and compare the same in Software Engineering, which is as creative a process as writing, you don’t see it at all. Which pieces of classic software that we study to learn about our craft or to inspire us?

In architecture and building construction, terms like ‘regeneration’, ‘revitalization’, ‘rebuild’ are considered positive terms conveying a positive message to the listener. However a common software term like ‘refactor’ doesn’t bring that same positive energy to a listener, especially if she’s not a Software professional. What has caused this? Is the insatiable thirst for shiny new things depriving the industry of the chance to look back and appreciate what was achieved before us?

Another factor for this immature attitude towards already created running software could be the on going high percentage of newcomers to the industry. It says that 50% of the industry is having less than 5 years of experience. This is a probably one reason why there’s a lack of love for existing systems and code. It’s hard to appreciate golden oldies when all you have seen is less than 5 years old.

Approaching Brownfield
I’ve been lucky to be in both sides of the green and brownfield development for the same application over past few years. I joined the team when the application was being written work for some time, and then left. And then joined the same team when the application is on maintenance mode. So today, while cringing at some of the code and patterns used in the code base, I soon realise that it’s probably me who wrote it couple of years back and then start empathising with that developer - who happens to be me in this case.

The point is, to be successful in brownfield you got to have empathy. You got to put yourself under numerous constraints the previous developer might have had to work under. The constraints could be his own lack of experience, time pressure, lack of functional clarity, technical limitations at the time etc…The common syndrome of this code sucks doesn’t quite work here - because I’ve come to realise that this attitude tends to extend beyond the code. May be you should focus on the code itself and get the individuals who are behind that code out of your mind.

Another good technique of approaching brownfield is to treat it as a sort of detective or investigative work. You have the remains of a case - code - and then you got to draw up whole crime scene. The motives and decisions that went in to the code at hand. The tools at your disposal could be unit tests (even if they turn out to be disposable), knowledge about technical limitations and best practices at the time or even personalities and work patterns of the developers involved in the code base.

The rewrite
A common pitfall of brownfield development is the desire to rewrite - rewrite from scratch. This is equally dangerous for a module or component as much as the whole system. There’s a lot of implicit knowledge hidden beneath that code than  you care to appreciate. A rewrite should be taken extremely cautiously, both for technical and business reasons. Most of the time, the men with the cheques don't care a hoot about whether the application is using jQuery 1.5 or 1.9, they just want the defects to be a minimum and system to be performant. So any considerable rewrite has to justify both the technical and business risks. Middle path here is to develop a separate application/module on top of the existing code base and just route the users to this new piece instead of the old one. The old one is still kept for sometime as a mitigation strategy. I’ve found that this strategy gives the business decision makers lot more peace of mind which actually increases the chances of a rewrite being approved in the first place.

Another opportunity to rewrite parts of the system is when a new piece of work (Enhancements) come along your way. This is the chance to get some free testing and strategically do some framework enhancements or component rewrites under the covers.

Adding value
Almost 95% of the industry is working on maintaining a system. So as an industry, it’s surprising that the amount of resources available for newcomers (Remember 50% of us have less than 5 years of experience) are primarily focused on developing systems from scratch. I hope there will be more resources on how you look after a live system and improve it while it’s kept running.

Probably the first improvement is to making sure that you use modern tools and techniques to look after this system. Again, if you look at the building industry you won’t see builders coming for a repair job of an old 1800 built classic building with just stone knives and hammers. They will have the latest tools. If the code base is not yet ready for that, the first job is to upgrade it so that it’s ready. An example from .Net would be to making sure that the code base is compatible with latest Visual Studio.

A low hanging improvement for these systems is to upgrade to the latest frameworks or to latest servers. Say for a web application, this could be upgrading from IIS6 to IIS8. Without changing any code the users could experience vast improvements.

Sometimes adding value to an existing system is misunderstood to making it look shiny. Although this could get the users attention, that’s probably not a high priority. If you are at early stages in a brownfield project, you might be too scared to propose big changes to the system - fair enough. Focus on what improvements you can do to make your teams life easier. How difficult it is to do a simple defect fix in a particular component. Try and make that component more welcoming to changes. May be you can write a test suite around that component inviting others to contribute and soon you realise that the team is not so afraid to change that component. Another thing is to review what version control the project uses and what kind of workflow is being followed. I’ve come to realise that some teams don’t get the best of version control and that makes them move at a lower velocity. It's not hard to find enterprise developers who had been bitten by using Source Safe and still don't quite get the benefits of having a modern version control system.

An ongoing issue in brownfield is how the implicit knowledge of your team members and code are captured. Do you have 5 year old functional specs that deviates from the current system so much making them useless or even dangerous? What works for our team is a Wiki. It’s a live document capturing bits and pieces of information that’s relevant on a daily basis. When documenting always try and stick to the ‘Whys’ - not ‘Hows’. Documenting why a certain decision has been made is far more important than documenting how a certain functionality works. The ‘How’ is already in the code - make sure your code is readable.

When adding a new feature or doing a significant enhancement always try and stay away from dependencies that are hard to change or obsolete. Instead try and replicate the old piece with new libraries or dependencies and run them in parallel. See how you perform over time and then you can even think about getting rid of the old piece. I've had to do this for some data access components which were dependent on an obsolete version of an Oracle driver. In the interim you might have a bit of duplication in the system, but you always have the long term goal in your mind. This is where software engineering purity doesn't quite get you there, pragmatism trumps purity in these cases.

Leaving Brownfield
Brownfield projects should not be treated as doing time for ones sins. It’s a great environment to learn from seasoned individuals and most importantly time tested and seasoned code base. Sometimes learning what not to do is as important as learning what to do.

Make sure you pick a few patterns and practices from the existing code base. Try and document them, since chances are you will forget these in a hurry. One way to do this is to update the existing documentation (wiki) of the project. This will add value to the team as well. Updating the Architecture document, starting a wish list, updating any existing defect records are activities that will leave a positive impression of you among  your team members.

Remember it will be so easy for the team (and quite natural as well) to blame you for some of the issues that could arise in the future after you are gone. Make sure that they don’t get that chance. Tidy up what ever little mess you have left unattended. Be it code, documentation or even a rogue folder in your dev server. Be nice to teammates and continue to value their time even after you leave.