Today is a glorious day for all C# developers that have been looking for a way to get away from windows as their development environment.
In my email today I got a lovely surprise:
Hi!
Xamarin has today announced Xamarin Studio, an evolution of the MonoDevelop IDE with tons of improvements. All the new code has been merged into the MonoDevelop repository and will be the foundation for future MonoDevelop and Xamarin Studio releases. If you want to try the new MonoDevelop/Xamarin Studio you can get installers for Mac and Windows from the MonoDevelop download page.
Enjoy!
Lluis.
I urge you all to give it a try. Here are some instructions so that you don't make the same mistake I made:
Download the latest mono for OSX. This is version 2.10.11
For years I wondered how do we know, if what we are
building is the right thing. In the development world we realised the
need for BDD
as we needed a way to rationalise our requirements and allow the
development team to know where to begin, however no methodology
assures you that your backlog is prioritised in the right order.
Don't get me wrong the Lean Startup is not a silver
bullet to the problem mention above. It is just that it got me
thinking about validating what we build.
"Lean Startup" is an approach for launching
businesses and products, that relies on validated learning,
scientific experimentation, and iterative product releases to shorten
product development cycles, measure progress, and gain valuable
customer feedback.
That translates to me rather than guessing or letting
emotion get in the way that we can build great products.
Minimum
Viable Product
The idea behind an MVP
is to build just enough to validate that we are heading in the right
direction. Some examples
of MVP could be:
“If Apple can launch a smartphone without Find or
Cut-and-Paste, what can you cut out of your product requirements?”
Well since we just want to validate we should reuse
as much of what other people have built. From a code perspective this
means it does not have to be perfect, however quality should not be
thrown out.
Continuous
Delivery
The book talks about Continuous Deployment, however I do
believe that the better phrase should be Continuous Delivery.
While continuous deployment implies continuous
delivery the converse is not true. Continuous delivery is about
putting the release schedule in the hands of the business, not in the
hands of IT. Implementing continuous delivery means making sure your
software is always production ready throughout its entire lifecycle –
that any build could potentially be released to users at the touch of
a button using a fully automated process in a matter of seconds or
minutes.
The point of the matter is that we want to get our MVP
out there as soon as possible. So from a software point of view we
really need to get our release cycle automated. I was never really a
fan of all these platforms that you just check in the code and it is
deployed on a live environment, this is because you really need to
understand how software is released. However to validate the MVP it
is really a great platform, so my thoughts are slowly changing.
Split
Testing
Split testing or A/B
Testing as it is sometimes called is a way to have two versions
of the application and validate which version is better. The way that
I have seen split testing being performed is that we have an existing
product and we split it with the new product that we are building.
However this got me thinking, shouldn’t we really be
doing split testing of a MVP in the same product?
So how would one go about doing some split testing in an
existing product. These are just some of my thoughts that I look
forward to validating.
To build these MVP's we need a way to create feature
branches easily. These feature branches should not be long
lived. The best tool for this is git.
The beauty of this is that if we realise our feature is crap we can
just revert
the feature.
The MVP needs to be build with a feature
flag. This feature needs to be turned on for a specific group of
people. This can be done in numerous ways, e.g. If it is a web
application your load balancer could introduce a HTTP header into
the application which would turn on the feature.
Lastly we need a way to gage if the MVP is worth it.
One needs to choose metrics
that matter.
Actionable
Metrics
The only metrics that I've been exposed to is what are
called vanity
metrics.
These are as an example: one million downloads, 10
million registered users, 200 million tweets per day.
These metrics show growth however they don't really tell
you the inside story. It is important to realise that we need to keep
the actionable
metrics closer to the user.
Actionable metrics can lead to informed business
decisions and subsequent action.
So how does this relate to the MVP? Well sometimes we
just want to get a sense if the feature is worth it, so just getting
an idea of whether people actually interact with it maybe enough.
Pivot
A pivot is a “structured course correction designed
to test a new fundamental hypothesis about the product, strategy, and
engine of growth.”
This basically tells us that at times we will have the
wrong idea and will need to change our direction. The key to be able
to pivot and abandon an MVP is to have easy way to revert the code
and release it quickly.
Conclusion
This book really made
me think about how to develop a product without incurring much waste
and as
a team be sure that we are building the right features.
For
years I had witnessed that the engineering practices have not been up
to par with what the rest of the industry was doing. As a development
team we weren't focusing on quality and every thing that we did was
very manual. This caused me great discomfort so I began preaching on
the values of agile and the practices from extreme programming that I
have had success with in the past. For a while I felt it was going
nowhere until certain key people were hired that made that journey
easier (you know who you are).
The
journey began early 2011 as we were faced with a difficult decision.
We had built an internal CMS that has met our needs for the last 10
or so years. However as with many software solutions, the years have
not been kind to it. This solution was basically built with one
purpose, to build web sites. However we were facing the dilemma of
multiple devices. Development was done through a text area and the
code was not version controlled. We unfortunately hit a brick wall
with it's capabilities. The question arose do we rebuild it or do
we look at paid product?
Previous
attempts to rebuild the CMS had pretty much answered that question,
so we decided to start searching for a product. We needed a product
that was extensible and was built using .NET
so we thought the best product for us would be Sitecore.
To be
successful from an engineering point of view, we wanted to accomplish
the following:
Build
a new website using a new CMS product.
Introduce
quality into the product using TDD
and BDD
This
was a very ambitious goal and a massive transformation.
We
already were using Scrum within other projects, however we all agreed
that this was not Scrum, more like mini waterfall. We were building
things in sprints and releasing months later (due to technical and
process issues). So we began to plan our first iteration.
Sprint
Zero
People
often underestimate the importance of sprint
zero.
We were one of these people. Don't get me wrong it's not like we just
jumped into it. Our sprint zero looked like this:
Get
a backlog of features that we needed to build.
Set
up a continuous integration environment along with version control.
Set
up a source code structure.
What
was the most important step we left out? We had not tackled the
problem of releasing our software. The issue that we had was that we
really didn't know where our solution would end up and we also knew
that a feature would not be complete any time soon, so we left it.
The lesson for me was, it is important to set up a process
that will deploy software. As code that is sitting in version control
is wasted.
Process
We
started off with Scrum and implemented the following routine:
Two
week sprints
Two
hour planning meeting at the beginning using planning poker
Our
board was modelled around the Lean approach of mapping
the value stream. We were also using LeanKit
to measure Lead
and Cycle Time. We later dropped LeanKit to use Jira
with
Greenhopper, however this turned to be more complicated that just
using LeanKit. I enjoyed measuring these times as I found that having
a conversation around statistics at your retrospective gives you a
great way to talk about continuous
improvement.
So
what was the end result? Well we started using more of a lean
approach using Scrumban.
We had weekly sprints and we didn't have long planning meetings. We
would try to raise issues as they appeared and put it on the board to
remind us to tackle it and we would still have retrospectives every
few weeks.
Story
Writing/Use Cases
We
were all big fans of User
Stories Applied. So we started to write stories in the following
format
As
a <role>, I want <goal/desire> so that <benefit>
During
the story writing session we realised that the last part of the story
was often left out and Mike Cohn says that it is optional. I started
to disagree, if you can't explain the reason is it really worth it? I
think this format would have been better
In
order to <receive benefit> as a <role>, I want
<goal/desire>
This format puts the benefit first which I think it is
important. We all know that a story is a promise for a conversation,
however this was not always the case and conversations were left out.
The acceptance criteria was captured using gherkin
language. Unfortunately this was easier said than done, as it was
new and people in general find it hard to describe the system. Due to
this the development team started writing the acceptance criteria.
Unfortunately to me this felt completely wrong. The reason is
as follows:
Usually the rest of the team would not review the
criteria (product owner, scrum master)
The language would turn into developer speak, which
means the rest of the team would not get it.
If the development team is going to write the specs I
find it easier to follow your trusted way of testing as the overhead
of the gherkin language is not worth it. One needs to remember
that BDD is a collaborative approach.
Testing
As I mentioned above we held quality very high as a
practice so we decided to have the following levels of testing:
Unit Testing
Integration Testing
Acceptance Testing
We had some challenges with the testing:
Unit testing is hard to achieve using Sitecore. So we
had to create a thin abstraction around it.
All
of Sitecore
is driven around configuration file (roughly 4000 lines). We had to
customise the config file to be able to do integration testing. This
proved that we weren't really testing the same system. Due
to this we had to create a matrix of the parts that we could test
and the parts that we should avoid, which is not ideal.
Acceptance tests were sometimes written to hit an API
and sometimes to hit the UI. This proved to be complicated as we now
had tests that looked like integration tests so the line was
blurred.
These challenges proved costly to us. The biggest cost
is that to be able to release a change the build takes over an hour
and some of our tests are flaky.
The
lesson here is to always pay attention to the testing
pyramid.
DevOps
This
area of our journey proved to be the most difficult.
The reason I say this is that
culture needs to change for DevOps to be successful.
We
are all big fans of Continuous
Delivery so we wanted to
make sure that we made this a reality. The biggest challenge that we
faced was actually having somewhere to deploy our application to.
Long story short we fought with some hosting providers and decided
that to really embrace continuous delivery we needed
to go to the cloud.
Here is
a summary of the tools that we used to make DevOps a reality:
Continuos
Integration started with Bamboo, however this moved to Team City
All
the scripting was done using
PowerShell. The reason for this is that our application is based on
.NET and being deployed on windows. In hindsight writing PowerShell
is easy, however maintaining a large codebase with tests and specs
is very primitive. Looking forward we would use Ruby.
We
implemented our own package manager that is based on nuget.
The package manager code is
based of the following code.
Moving forward I would
consider
to use Puppet
All
of our instances are prepared using Puppet and the initial
infrastructure is set up using CloudFormation.
As mentioned
previously our biggest challenge was to bring the two teams together.
Some of the things that I found were as follows:
Not everyone
in operations believes they should be doing infrastructure as code.
Not
everyone in the development team cares about infrastructure as code
and understanding
infrastructure.
Due to
the above reasons it is tempting for organisations
to create a DevOps team. I
don't agree with this as it is important for the organisation to come
up with their definition of DevOps.
If you know you don't have
the right
fit then go ahead and find it!
Conclusion
We
had a great journey and have learnt so much about Agile, Continuous
Delivery and DevOps. It
wasn't
always a smooth ride however we were able to pull it off and have
lots of things that we can improve. Some areas that we want to
concentrate on are:
So about 6 months ago I submitted a patch to try to fix the out of memory exception that we were getting with NuGet, due to having large packages.
The reason I discovered this was because I wrote a tool that emulated apt-get using NuGet that is on my github. I really saw the potential with using nuget for .NET projects.
Unfortunately my patch got rejected after a 6 month code review with the following message:
To update everyone on this issue, I'm planning to reduce the overall usage of MemoryStream in NuGet.Core. In many cases, we load the package contents into memory unnecessarily, hence the OOM exception. It will address the issue better than trying to band-aid it with writing to a temporary file, which could potentially affect perf. I'm sorry to decline the pull request, but rest assured that the OOM exception will get fixed in 2.3.
So why the disappointment? Well rather than work in a open source way Microsoft decides that they can do it better. It is just sad that this software giant does not understand OSS.
The DevOps movement has been an interesting and confusing term in
our industry. I thought I would write about this because I had an
interesting conversation at work and it seems that we all had
different ideas of what this lovely word means.
DevOps is a software development method that stresses
communication, collaboration and integration between software
developers and information technology professionals. DevOps is a
response to the interdependence of software development and IT
operations. It aims to help an organisation rapidly produce software
products and services
So what does this mean?
DevOps is purely a culture, a way of thinking and interacting.
The biggest confusion that I saw with my peers is that DevOps is a
team that needs to be formed as a separate entity. This is wrong!
DevOps is collaboration between people
from the development, operations and testing. This gathering is
controlled purely by the nature of a self
organising team. They don't report to a DevOps manager.
Another misconception that I noticed
was that Infrastructure
As Code is DevOps. This is wrong!
Infrastructure As Code came from
the understanding that people were tired of having Snowflake
Infrastructure and a lot
of these problems have been solved in the development
world. This really means
that we are being pragmatic and solving our own
problems by learning from each other.
Infrastructure As Code is the key to
implementing DevOps concepts.
Hopefully this clarifies some of
the misunderstandings, you can find further information about
DevOps can be found here.
A
while back I got introduced to the concept of functional
programming and like most of you out there I found it hard to
understand some of the concepts. Over time I have slowly started to
understand some of the concepts and thought I would try to once for
all capture them as I have a memory of a goldfish. Please
note that it is important to try to understand these concepts if you
are going to understand this blog post.
A
programming paradigm that treats computation as the evaluation of
mathematical functions and avoids state and mutable data. It
emphasises the application of functions, in contrast to the
imperative programming style, which emphasises changes in state.
Functional Programming is defined as using a style
called Declarative programming rather than your traditional
Imperative programming. What do these mean styles mean?
A
programming paradigm that expresses the logic of a computation
without describing its control flow. Many languages applying this
style attempt to minimise or eliminate side effects by describing
what the program should accomplish, rather than describing how to go
about accomplishing it.
In
computer science, imperative programming is a programming paradigm
that describes computation in terms of statements that change a
program state. In much the same way that imperative mood in natural
languages expresses commands to take action, imperative programs
define sequences of commands for the computer to perform.
The
following points are what some of the functional programming
languages exhibits:
Immutable
Object
Higher
Order Functions
Currying
Lazy
Evaluation
Continuations
Pattern
Matching
Closures
Luckily
for us is that
PowerShell
is motivated by functional programming
Right so lets go through these points and how they apply
to PowerShell.
Immutable
Object
An
immutable object is an object whose state cannot be modified after it
is created. This is accomplished as follows:
By
adding a property that is of type ScriptProperty it generates a
getter. When we try to modify it it will throw a
System.Management.Automation.SetValueException. The example uses a
Hashtable as the input, however it is quite easy to extend it to use
any type of object.
Higher
Order Functions
A
higher-order function is a function that does at least one of the
following:
Take
one or more functions as an input
Output
a function
So
lets look at a simple test that will work out even and odd values in
an array:
PowerShell
has a concept of a script
block. The important thing to notice is that we are using a where
expression and we are invoking the predicate with an & (this took
me a while to work out).
Currying
Currying
is the technique of transforming a function that takes multiple
arguments (or an n-tuple of arguments) in such a way that it can be
called as a chain of functions each with a single argument.
This
has proven to be a bit difficult in PowerShell however it is not
impossible. Let's look at the example below:
As we
can see, a curried function can be created as an anonymous function
or a named one. I unfortunately could not find a clean way to
represent this as a generic solution. The trick here is to always
return a new closure. There is a JavaScript
implementation that might be able to be used in PowerShell that maybe
someone could help me with.
Lazy
Evaluation
Lazy
Evaluation is an evaluation strategy which delays the evaluation of
an expression until its value is needed and which also avoids
repeated evaluations.
This
fortunately is easy to create as the wonderful people in the .NET
framework created the class System.Lazy.
Lets have a look at some PowerShell code:
Describe "Lazy" {
$lazy = New-Lazy { return "test" }
It "Should not have a value evaluated" {
$lazy.IsValueCreated.Should.Be($false)
}
It "Should get lazy value" {
$lazy.Value.Should.Be("test")
$lazy.IsValueCreated.Should.Be($true)
}
}
function New-Lazy($script) {
$function = [System.Func[object]] $script
$lazy = New-Object System.Lazy[object] $function
return $lazy
}
All
we need to do is create a Lazy object and call the Value property.
The implementation will cache the value.
Continuations
A
continuation is an abstract representation of the control state of a
computer program. What does that mean?
This to me was complicated to understand why would you
want such complication. I found a great explanation in the following
article
of how you can use continuations in a real world sense (nothing worst
than thinking in abstract terms)
PowerShell does not have a feature like this (neither
does .NET for that matter). The closest thing that PowerShell has is
what is called Workflows.
To look at how these are implemented
I will need another blog post.
A
function written in continuation-passing style takes as an extra
argument: an explicit "continuation" i.e. a function of one
argument. When the CPS function has computed its result value, it
"returns" it by calling the continuation function with this
value as the argument.
Pattern
matching is the act of checking some sequence of tokens for the
presence of the constituents of some pattern. Pattern matching in
PowerShell is accomplished using the switch
statement.
Using
your traditional imperative programming languages the switch
statement is considered a code
smell. The reason that they have been frowned upon it is simply
because they usually display signs of code duplication.
Pattern matching is encouraged in functional programming
languages because they are much more powerful and functions are first
class citizens.
Luckily for us PowerShell has a strong pattern matching
construct in the switch statement. Lets look at some examples:
Describe "Pattern Matching" {
It "Should use a simple switch" {
$a = 5
$result = switch ($a) {
1 {"The colour is red."}
2 {"The colour is blue."}
3 {"The colour is green."}
4 {"The colour is yellow."}
5 {"The colour is orange."}
6 {"The colour is purple."}
7 {"The colour is pink."}
8 {"The colour is brown."}
default {"The colour could not be determined."}
}
$result.Should.Be("The colour is orange.")
}
It "Should use a wildcard switch" {
$a = "d14151"
$result = switch -wildcard ($a) {
"a*" {"The colour is red."}
"b*" {"The colour is blue."}
"c*" {"The colour is green."}
"d*" {"The colour is yellow."}
"e*" {"The colour is orange."}
"f*" {"The colour is purple."}
"g*" {"The colour is pink."}
"h*" {"The colour is brown."}
default {"The colour could not be determined."}
}
$result.Should.Be("The colour is yellow.")
}
It "Should use a regex switch" {
$a = "r14151"
$result = switch -regex ($a) {
"[a-d]" {"The colour is red."}
"[e-g]" {"The colour is blue."}
"[h-k]" {"The colour is green."}
"[l-o]" {"The colour is yellow."}
"[p-s]" {"The colour is orange."}
"[t-v]" {"The colour is purple."}
"[w-y]" {"The colour is pink."}
"[z]" {"The colour is brown."}
default {"The colour could not be determined."}
}
$result.Should.Be("The colour is orange.")
}
}
As we can see this is quite powerful as the switch
statement returns an expression.
Closures
A closure (also lexical closure or function closure) is
a function together with a referencing environment for the non-local
variables of that function.
Closures
in PowerShell are created with the function GetNewClosure.
The
new script block is closed over the local variables in the scope that
the closure is defined in. In other words, the current values of the
local variables are captured and enclosed inside the script block
bound to the module.
You have seen this in the previous examples.
Hopefully this has been an interesting journey through
some of the functional programming concepts and how they can be
applied to PowerShell. I will try to build a module that makes it
easier to use some of theses constructs.
You cannot depend on your eyes when your imagination is out of focus. - Mark Twain
I am a big fan of PowerShell, this is a rare occasion that Microsoft gets something right. If you want to learn more about PowerShell I suggest you buy this really great book or follow this blog. I thought I would start a series that outline some interesting and wonderful things I find about the language.
Today one of my colleges at work asked a very wonderful question about scopes, which left me puzzled (he is a JavaScript guru so I didn't understand why he didn't get it ;)
If you read through the official documentation about scopes you might get a bit confused like I did.
Local:
The current scope. The local scope can be the global
scope or any other scope. -- Hmmm
As we can see from the documentation PowerShell does not mention a concept of function scope like in JavaScript. If we take the JavaScript example and turn it into PowerShell we might get a surprise
The $player variable is still $null. Not what I expected. However if we define a function within a function then the variable is available in the inner scope.
$sport = "baseball"
$player = $null
function Get-Player() {
if ($sport -eq "baseball") {
$player = "Evan Longoria"; # (the baseball player)
}
else {
$player = "Eva Longoria"; # (the actress)
}
$player2 = "Derek Jeter";
function Inner-Function {
Write-Host $player
Write-Host $player2
}
Inner-Function
return $player;
}
Get-Player
If we prefix the $player2 variable with $script it is now in scope.
