Thoughts on inaugural #c9d9 online discussion panel

On Wednesday I took part in an online discussion on Agile / Continuous Integration / DevOps / Continuous Delivery organised by Electric Cloud (see http://electric-cloud.com/blog/2014/10/c9d9-continuous-discussions-episode-1-recap/) .

I was really happy to be invited to talk as a panellist on the discussion, especially seeing the very high calibre of the other individuals on the panel.  It was great to be included among such an insightful and talented group of people.

One of the things that struck me was that there was widespread agreement among the panellists of the benefits of Agile / CI / DevOps / CD, and many of the barriers that were encountered were also the same.

In the programme I work on at the moment I feel that we have quite a slick delivery machine that incorporates:

• Development - making code changes
• Automated CI build and scheduled build
• "Touch of a button" scripted tear down / rebuild of test environments (both functional and performance), with full redeploy or upgrade of the software
• Comprehensive integration tests
• Ability to "ship" as soon as the software on test has passed QA

However, within all this goodness there are some significant barriers.  I don't want to beat up my end customer too much, because I understand that if my software goes down for any length of time they're pretty much out of business.  It's a big responsibility.  However......

• The end customer is ultimately responsible for integrating, final QA and release into production.  The point was made on the discussion panel that if you look at your development process as a pipeline you can only move at the speed of the slowest stage.  This is definitely an issue where we are at the moment, we have the ability to produce much more value but we are constrained because the end of the pipe cannot absorb change quickly enough.
• As a consequence of this, as releases become further and further apart they become larger and hence have a higher level of inherent risk.  It is an irony that the risk-aversion that leads to endless test cycles of an integrated solution ends up slowing down the deployments so much that each deployment becomes ever riskier.
• One of the keys to solving this is DevOps.  Within my dev programme we've got great build and infrastructure guys embedded so we can all work together to create our slick process.  At the end customer the various teams involved are all in different organisational silos (and often in different organisations altogether).  The multi-functional teams required to make releases slick and frequent are not in place.  
• This makes for an interesting observation.  In software, quality comes from automation, because only by using automation can we be truly repeatable, and a precondition of software quality is repeatability.  But - again an excellent observation made on the panel - automation also frees software developers from mundane repetitive tasks so that they can focus on spending more time delivering solutions.  We should embrace automation, it is our friend.
• Also, and this is my own reflection on this, true agility (as opposed to agile methodologies, which I prefer to regard as strategies for managing change) comes from adopting best practices.  It involves setting up the software development / test machine (note that these two cannot be separated) so that when we change the solution we get feedback on the level of quality quickly. As anyone with a knowledge of control circuits knows: if you want a stable system the speed of change has to be slower than the speed of feedback .  
• Which brings me back to where I was before.  Long release cycles involve slow feedback.  We try to get around this by delivering into a QA environment on several-times-a-day basis (fast feedback).  However, we are at the mercy of integration issues at the other end of the pipeline and there the feedback can be months.  This causes us issues because we are regularly working on updated requirements to fix integration issues - not because this adds more business value, but because slow integration did not catch this early in the cycle.

A great discussion, much food for thought for me about where I want to take this.  My current programme is winding down at the moment, and I'm thinking of what I want to work on next year.  All I know for sure is that I want MORE CI, MORE DevOps and MORE Continuous Delivery!

And to work on big stuff.

Many thanks to everyone involved in #c9d9.  

Details of next month's online discussion on Continuous Delivery

*I'm a panellist on the first "Continuous Discussion" community panel - you're invited!*

On October 8th I'll be joining an online panel of experts and practitioners in Agile development, DevOps, CI and Continuous Delivery, hosted by Electric Cloud who have done large Continuous Delivery projects for organizations like SpaceX, Cisco, GE and E*TRADE. Other panelists include J. Randall Hunt, evangelist at AWS, Sriram Narayan, IT principal at ThoughtWorks, and Carlos Sanchez, Apache member and frequent speaker on CD/Devops.

In the online panel, myself and the other panellists will run through a few topics which relate to hands-on implementation of these methodologies in various technology and business environments. We'll share our real-life experiences, challenges and quick wins. It will be a "grass roots" discussion of Continuous Delivery and how/to what extent we can implement it.

From my perspective I'd like to recall a few of my own experiences that highlight the need for good DevOps and Continous Integration.  There have been some moments of disaster, some moments of success, some moments of epiphany.  

Also, having worked on the same programme for the last 5 years I've been quite insulated from the outside world, and I'd really like to find out more about what other people are doing out there to improve the software development practices.

Join me at the online panel on October 8, 6-7pm BST.

Please follow this link for more details from the organisers:
http://electric-cloud.com/blog/2014/09/continuous-discussion-online-panel/

Add to your calendar:
http://electric-cloud.com/wp-content/uploads/2014/09/Continuous-Discussion-1-Community-Panel.ics

Got any questions, tips or things you’d like me to address/cover in the panel relating to Agile, CI, DevOps or continuous Delivery? – don’t hesitate to comment or message me!

Time to Start Speaking Again!

I've been heads-down on project work for so long now that I haven't really got round to posting thoughts and sharing things I've learned.  I've decided that it's time to change this now.

I'm really excited that I've been invited to talk on a panel about Agile, DevOps and Continuous Delivery, which are areas of my job that I've been trying to get better at for years and am still learning.

I will share more details of this session when they are confirmed, but it's on October 8th at 10am PST (6pm in the UK).  In the meantime I'll get some more thoughts together on what I've found interesting lately.

Using Team Foundation Server 2010 from Visual Studio 2005 and 2008

On my project the TFS server has just been upgraded, ready for us to replatform the software onto .Net 4.0. In the meantime we're still working on a mixture of Visual Studio 2005 (for BizTalk 2006 R2) and Visual Studio 2008 (for the rest of the .Net and database code).

Team Foundation Server 2010 is not compatible with Visual Studio / Team Explorer 2005 and 2008 without the compatibility upgrade.

In order to install the compatibility pack for Visual Studio 2008 go here:

http://www.microsoft.com/downloads/details.aspx?FamilyID=CF13EA45-D17B-4EDC-8E6C-6C5B208EC54D&displaylang=en

In order to install the compatibility for Visual Studio 2005 go here:

http://www.microsoft.com/downloads/details.aspx?familyid=22215E4C-AF6F-4E2F-96DF-20E94D762689&displaylang=en

Workspace issues

Once you have connected to the new TFS server and mapped your projects back, you should be able to map your workspaces without any problems.

However, you might find that your workspaces do not work. This is especially an issue in Visual Studio 2005, which seems not to recognise the workspace that you have reapped the workspace. If this is the case, run the following command from the VS2005 Command Prompt:

tf.exe workspaces /remove:*

Then get latest on your source branch. It should be OK.

The endless debate on "what is an architect"

Just been following the following debate in LinkedIn: http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers&discussionID=13839089&gid=100988&commentID=12211252&trk=view_disc

The discussion started with the following comment:

This is what is killing Enterprise Architecture…
Quote from "John Zachman" article..;-)
This is what is killing Enterprise Architecture… every computer programmer, systems designer, software architect, solutions architect, technology architect, computer operator, PC owner, data architect, database architect, network architect, business analyst, systems analyst, enterprise architect, service architect, object architect, project manager and CIO calls whatever they want to or maybe, whatever they are doing, “Architecture.” It is chaos. No wonder we don’t have Enterprises that are coherent, integrated, flexible, dynamic, interoperable, reusable, aligned, lean and mean and working.

And I thought I'd add my bit as well.....

I love this discussion, if only because it goes round in circles so much. Getting back to the original point though:

"....calls whatever they want to or maybe, whatever they are doing, “Architecture.”"

This gets to the nub of the problem. Quite simply, you are an architect if the role you perform is "architecture". However, people do indeed hijack the term to make their own role sound better. We cannot rely on some higher authority to define for us what an architect is and then impose a set of criteria to test against. Instead it is incumbent upon all of us who consider ourselves to be architects to explain what architecture involves and if we suspect others of "job title inflation" we should ask what they have done that qualifies as architecture.

As a marker, I think there are some fundamental aspects of architecture that we should be able to agree on:

* Architecture is a high-level design activity. The output of architecture is a design of some sort, although this can vary enormously in the level of abstraction. Architecture is the synthesis of a design that satisfies a set of requirements.

* A secondary activity of an architect is communication of the design and the supervision of the detailed design and implementation to ensure that the design intent has been captured.

* Another secondary activity of an architect is to communicate to those creating requirements what the impact of their requirements will be on the design and to advise on improvements.

Someone whose primary role is development / implementation / operations is not therefore an architect. I think "enterprise" architecture vs other types of architecture is a red herring in this regard. These are all valid disciplines - there are indeed "data architects", there are indeed "infrastructure architects" and there are indeed "enterprise architects". The key thing is that in each case the primary goal of their activity is design.

You've got to love it. I haven't seen a more circular debate since the old artists vs scientists one that you have at university!

User-Defined Functions for Today and Tomorrow in SQL

Just a quick post today. I have been working on some bugs on a system were about to release, which was originally built by some dodgy developer before we took it over. The older reaches of the application are a total mishmash f things that should be in the database, data access layer, business logic layer and in the UI, so please don't flame me with comments like "you shouldn't be doing this in the database, it should be in the ......".

One of the features that I was fixing was a view that contains the current effective price for a product. Imagine that you have a table that contains prices, and when a price changes you insert a new row with the updated price and an effective date. Simple stuff - the current price is the one with the most recent effective date that is either today or in the past, i.e. before tomorrow.

Now, SQL server is a pain in the ass when it comes to simple date stuff like this, because all I want is something like this:

SELECT

TOP 1 *

FROM

ProductPrice

WHERE

ProductID = @ProductID

AND EffectiveDate <>

ORDER BY

EffectiveDate DESC

I therefore wrote a couple of functions to achieve this:

CREATE FUNCTION [dbo].[Today]()

RETURNS datetime

AS

BEGIN

DECLARE @today datetime;

SET @today = convert(datetime, convert(varchar(10),getdate(),101), 101);

RETURN @today;

END

CREATE FUNCTION [dbo].[Tomorrow]()

RETURNS datetime

AS

BEGIN

DECLARE @tomorrow datetime;

SET @tomorrow = convert(datetime, convert(varchar(10),getdate(),101), 101);

SET @tomorrow = DATEADD(d, 1, @tomorrow);

RETURN @tomorrow;

END

Enjoy :)