TheTrav

I'm not a cylon!

0 notes &

Play Json Combinator Confusion

validate[Option[String]](OptionReads(myCustomStringValidator))

if the validator returns JsError, the OptionReads returns None.
Quite unexpected, I’d almost go so far as to say wrong.

validate[Option[String]]((__).readNullable(myCustomStringValidator))

If the validator returns JsError the OptionReads returns JsError.
Expected behaviour, but I’m at a loss to explain why the two should behave differently.

0 notes &

Playframework and Thread Safety

Having some “fun” with playframework of late.

FluentLenium

I noticed with 2.2 they introduced the WithBrowser context for doing FluentLenium tests.

I have a long running argument with myself over whether browser based end to end testing is worth the time and maintenance.  

Ultimately I’ve settled on the decision that for the 5% of your app that is happy path critical functionality, it can show up a lot of wiring problems that your other tests won’t

Parallel Specs

By default play uses akka to run as many of your tests in parallel as it can.  Thus you get a new instance of your “FakeApplication” for every WithBrowser context.  

This is probably ideal as you don’t want them clobbering each others data.

Unfortunately however, at the end of the day these FakeApplication instances all run on the same JVM.

Controllers are Objects

Objects are static singletons.  The JVM doesn’t care that you’re using multiple threads, it’s going to initialise those controllers once, and once only, your vals from your first test will be the same vals for your last test, and all the ones in between.

Configuration

So finally, the bit I ran into.  each WithBrowser piece has the opportunity to pass through a custom FakeApplication, with a custom additionalConfiguration.

The methods for pulling config out of the current application is fine, because it’s a def, which will be executed every time you want it.

Unfortunately if you have a DRY config, and use string templating for a lot of your end points, you’ll tend to write code like this:

    val baseUrl = Play.current.configuration.getString("baseUrl").getOrElse("localhost")
    val serviceUrl = s"http://$baseUrl/servicePath"

Which is great for DRY, but it means all your tests have to be ok with the same endpoint urls.

The Work around

Personally, I went through my app and changed all the val’s that held configuration data into defs.

This works, however it relies on eternal vigilance, and if you mess it up, you’ll get unexpected errors.  

I’m not sure if it’s exactly a timing/ordering issue as I seem to get quite consistent failures when I’ve got it set up incorrectly, but it’s easy to make a change that shifts the ordering, and if you don’t know what the problem is, you get a success leaving the trap for the next poor sucker.

The Theory

So, another idea that I’ve been toying with is forcing all my objects (controllers, global settings, and filters maybe?) to new up something to do any actual work.

It’s still eternal vigilance, but it pushes the problem into a single layer which might be easier to keep an eye on.

Ultimately I think it’s just going to be something you need to be aware of when working in a multi threaded environment in scala.

Filed under scala play framework

0 notes &

Play Framework 2.1 Scala Controller Unit Test with Mockito

FakeApplication sucks

Play 2.x does a lot to make writing controllers a breeze.

Unfortunately a lot of their choices can lead you down a path that discourages unit testing.

The official docs recommend using FakeApplication to cover controllers, however this comes with many shortcomings, the most notable an increase in test execution time to the tune of 2 orders of magnitude!

Delicious Caek

Andrew shows you how to solve this problem using Guice.
I’m not such a fan of annotations, so on our project at APDM, we’ve gone with the cake pattern and mockito to get our test coverage.

To start with, lets build a service:

trait MailServiceComp {
  val mailService: MailService
  class MailService {
    def all = {
      List("normally", "real")
    }
  }
}

We have all our services binded in a Context class, this gives all our controllers access to all our services with a single mixin.
Either a blessing or a curse depending on your point of view, but that’s another blog post

trait Context extends MailServiceComp {
  val mailService = new MailService
}

First contact

Now lets build our controller to use this service.
Lets assume I’ve already built the list template in /views/mail/list.scala.html

object Mail extends Controller with Context {
  def list = Action {
    Ok(views.html.mail.list(mailService.all))
  }
}

So this is incredibly conscise, and full of type safe goodness, however when it comes time to test, we’re going to have a hard time stubbing the service.

Moar Caek!

I still don’t have the lingo down, so I’m actually not sure if it counts as cake pattern without self typed references… This may just be plain old inheritance, but the same principle applies

object Mail extends MailController

trait MailController extends Controller with Context {
  def list = Action {
    Ok(views.html.mail.list(mailService.all))
  }
}

This separation of the trait from the object lets us write a test. First however:

Dependencies

At this point I think it’s important to warn you about a few things.
While play is bundled with specs2, and specs2 is bundled with Mockito bindings, as of play 2.1, neither has Mockito as a dependency.
Therefore in order to use those bindings, you need to add

"org.mockito" % "mockito-all" % "1.9.5"

to the appDependencies Seq in your Build.scala, otherwise you will get something like:

    bad symbolic reference. A signature in MocksCreation.class refers to type MockSettings in package org.mockito which is not available.
  

Imports

There is also two imports you’ll need to have, it drove me crazy trying to find them.

import play.api.test.Helpers._
import org.specs2.mock.Mockito

The purpose of org.specs2.mock.Mockito is obvious, play.api.test.Helpers._ gives you the utility functions for extracting things from Result

Drumroll

class MailSpec extends Specification with Mockito {  
  "Mail" should {    
    "render our list" in {
      trait FakeContext extends Context {
        override val mailService = mock[MailService]
      }
      val mail = new MailController with FakeContext
      mail.mailService.all returns List("Test")

      val result = mail.list(FakeRequest())
      status(result) must beEqualTo(200)
      contentAsString(result) must contain("Test")
    }
  }
}

Now you have a controller test that doesn’t use FakeApplication.If you stop there, you’ve at least achieved a controller test in isolation from FakeApplication, which gives you:

  • Two orders of magnitude improvement in controller test execution time
  • Test service layer independantly from Controller and Views

A bridge not too far enough

The smell still remains in the above test, that to test the controller you also have to test your entire template structure.
For many systems the templates can be hundreds of lines across many files with a non trivial assembly.

To properly test the controller in isolation, I personally like to pass in the template function as a constructor arg. You can find the type for the templates apply function in target/scala-{ver}/src_managed/main/views/{path}/{name}.template.scala
It’s pretty simple, the parameter list you declare => play.api.templates.HTML So now, the constructor looks like

 object Mail extends MailController(views.html.mail.list.apply _)

class MailController(template: (List[String]) => Html) extends Controller with Context {
  def list = Action {
    Ok(template(mailService.all))
  }
}

It’s not quite the cake pattern, but given it’s a one off binding, that we’re really not going to re-use anywhere, I think cake would be overkill.

The test can now look like:

class MailSpec extends Specification with Mockito {  
  "Mail" should {    
    "render our list" in {
      trait FakeContext extends Context {
        override val mailService = mock[MailService]
      }
      val template = mock[(List[String]) => Html]
      val mail = new MailController(template) with FakeContext
      val serviceResult = List("Test")
      mail.mailService.all returns serviceResult

      val result = mail.list(FakeRequest())

      status(result) must beEqualTo(200)
      there was one(template).apply(serviceResult)
    }
  }
}

Navel Gazing

At this point you’ve probably noticed that you’re effectively testing the plumbing.
This is correct. Furthermore, plumbing is all your controller _should_ ever be.
In reality however, project deadlines have a habit of slipping more and more pieces of code into your controllers. This technique gives you a way to at least test those extra bits without paying the performance cost of spinning up a FakeApplication and without adding the complication of your entire templating tree to deal with

It is however, still important to remember to write tests on the other side of those mocks.

Play gives you a great structure for testing templates directly, and the services should be isolatable using cake.

Filed under play framework Scala testing mockito code

0 notes &

The war against estimation is harmful to developers

I was notified the other day of a discussion on estimates that flicked a few nerves.

The discussion surrounded eliminating points from the estimation process, attempting to break cards down to equal sizes.

I can understand the appeal, assuming everything averages out, however I don’t think it’s possible really to break down the cards to equal size chunks, and if you have varied sized cards, you lose the best thing to come out of agile methodology.

The separation between COMPLEXITY estimation, and TIME estimation. (I’ve ranted about that before)

There are two primary arguments that come out of developers who don’t want to estimate.

1 - I don’t want to commit to an estimate, and if I give you one I am committed

2 - If you want a number from me you don’t trust me to do the work

Both of these arguments are demonstrably false.  Additionally, they are somewhat contradictory, but often held together.  

If you don’t trust your ability to hit an estimate, then why do you expect your client to trust your ability to do all the work they require in the budget they have?

It’s a response to the fear of contract heavy development where planners don’t trust developers that swings the pendulum to the complete opposite direction, where developers don’t trust planners and refuse to allow them to plan.

It’s fairly self evident that it won’t work.  Any significant development will need some planning to happen around release dates.  This planning will be done whether you chose to engage in it or not.

By delivering accurate estimates, which are regularly checked, you are actually making it less likely that unreasonable delivery dates will be targeted, and making the discussion around moving them a whole lot more accessible and scientific.

Personally, I’ve lately been on a project where estimation was largely dropped for about 3 months.  

While we did deliver _something_ along dates which were presented to us, the process was not pleasant for anyone involved.  

There were many overtime hours and weekends worked (no way of measuring their positive or negative impact) there were many hurt feelings and complaints every time a new date or requirement came down.

The team had a fantastic level of skill, and the delivered product was a success, however there was not the sense of achievement and satisfaction you might expect after doing so.  In general people seemed to think we could have done a lot better.

Contrast this with my latest project, where estimation and regular analysis has been kept up for the 9 month start to launch.

The scope was bumped up by about 50/60% twice, and the team was doubled each time.

The original launch date was pushed out by 3-6 months.

There was of course, pressure to increase velocity and discussions were had, however they were accompanied by scientific data, the tone was never “you are not doing enough” because what we were doing was clearly visible in the burn up chart.

Once the planners moved the release date to account for the new scope and the achieved velocity, the project continued, as planned, and delivered a smooth release without any over time and while some people got a bit stressed in the final week, every single person I spoke to afterwards was satisfied with the result and happy with their achievements.

Now both of those situations were environments where the client already bought into the “agile” mindset, they were both reasonably flexible and had the ability to break away from “planned work” and focus on “delivering value”

I have been on at least two projects previously with pushy, borderline hostile clients, with less ability to shift from the established plan towards delivering real value.

In both projects I can think of, where estimations and velocity were tracked, the conversations over what gets delivered were framed in scientific reality, which goes a great length to cut through the political argy bargy of ‘you were contracted to deliver x dot points by y date’.

The problem seems to boil down to people believing that reasonable project planning, is on one side of an axis, with accurate developer predictions being on the other side.

It’s a false dichotomy, the two are in no way mutually exclusive.

Filed under agile estimation software development

0 notes &

Something that bugs me in Ruby

Disclaimer:  I would probably start most new projects these days in rails, the amount of current problems that have been solved and the almost seamless way of integrating those solutions is an incredibly compelling offer.  The community is really rails and ruby’s strength.

Having done quite a bit in ruby so far, I’m finding more and more that I miss my type system and get annoyed by the kludginess of everything I’m using.

Hashes, Symbols, OpenStructs

<pre>

structure = OpenStruct.new({“foo” => 1, :foo => 1, :children => {:tim => “mah”}})

structure.children.tim

</pre>

Mixing and matching symbols is a royal PITA.  
If you’re working with a web framework, at some point it will convert all your symbols to strings, then you’ll think about symbolize keys, but that won’t do the whole tree if you’ve got a nested hash.

Addressing nested structures in hash format is ugly hash[“a”][“b”][“c”] and although OpenStruct makes this easier, converting between OpenStructs and Hashes is painful with nesting, not seamless.

I find hash rockets to be clunky and annoying syntax.  The newer {symbol: value} format is much nicer, but what if I don’t want to use symbols for my keys?

IMO Maps are a really important part of every language, and it bothers me greatly that in ruby they are not terribly fun to use

Filed under ruby code dev

0 notes &

git subtree merge

I don’t really like git submodules.

It makes sense for sharing modules across teams / Open Source groups, but within a single project, even bigger ones, I’m overwhelmingly in favour of single repos with sub folders.

I sometimes see people in a setup where they have already got lots of little repos though, and they don’t want to lose their commit history.

Here’s how you merge them but keep the history:

mkdir target
cd target
git init
touch .gitignore
git add .gitignore
git commit -m "initial commit"

git remote add -f first ssh://dius@git-server/git/lots_of_repos/first
git merge -s ours --no-commit first/master
git read-tree --prefix=first/ -u first/master
git commit -m "Subtree merged in first"

git remote add -f second ssh://dius@git-server/git/lots_of_repos/second
git merge -s ours --no-commit second/master
git read-tree --prefix=second/ -u second/master
git commit -m "Subtree merged in second"
At the end of that you’ve got a new repo called target, with the subfolders first and second and all the commit history of the first repo as well as the commit history of the second repo.
Unfortunately because you’ve changed the folder path, if you do a diff between new versions and versions before the merge you get “everything has changed”, however diffs between multiple versions before the merge work, you also get all the orphaned commits from branches that you hadn’t merged back to master etc in there.

Filed under git dev version control

0 notes &

Technology Estimate Profiles

Today I found myself in a discussion about technology choice and estimates.

Given the current nature of the market, frequent small projects, lots of push for new technologies, this is a common situation to be in.

We had looked at the problem, come up with a solution, done a bit of a story by story point by point break down and come up with a proposal.

As is a common story, we chose what we feel is the best tech stack for the job, however this popular tech stack is not what’s used in the clients organisation currently.

The client wants to try new technologies, has heard of the one we suggest, but is nervous and wants it compared against their existing tech stack in every way possible.

The suggestion came up to supply a percentage increase in cost for using the existing, slightly older tech stack.

This is where things get a little murky.  I had quite a bit of difficulty explaining why I thought just adding a percentage and running with it was a bad idea.

I’ve been playing with OmniGraffle a bit lately, so during the meeting a bit of a diagram popped into my head to help me explain it.

The problem is in relative complexity.

Here’s my first awesome diagram

image

Pretend frameworks, pretend stories, pretend everything.  I’m illustrating a point here, not presenting evidence.  Certainly not buying into the group think of bashing one particular technology and glorifying another (undeservedly, but that’s another rant)

Pretend blue is a 1, yellow a 3 and red an 8.

Note how you could take a 1 from the emerald framework, drop it in sunacle, and it could be a 1, it could be a 3 or it could be an 8.

Why is this so?  Well, there’s tons of reasons.  A particularly compelling, and easy to understand one, is the footprint of changes on the system.

image

Different graph of the same fake stories, this time, showing the layers of the system that need to be touched to achieve each user story.

Notice how, in the upper framework, there’s lots of layers, and little is left to convention in the layers.  Lots of things need doing to affect even small changes.

In the other system, some layers aren’t used, and for many changes, only a few layers need to be touched just to specify some unusual departure from convention.

The important thing to note though, is that it’s not linear across the system.
To make up an example, changing a bit of static text on the view templating layer is likely to remain fairly similar across both frameworks.

Add a field to the database and send an email however… well, in one system it’s close to as simple as the text change, in the other it’s a major endeavour.

I like to call this a technology profile.  Some tech stacks are simply better at some things than others.  This comes out when you do proper estimates across both systems.  If you try to take the estimate from one technology, and linearly extrapolate it onto another, you’re going to come unstuck.  

You can take a %ge number across the project, and come out with approximately the correct total.  

However if you do that, there’s a reasonable chance that the relative complexities you came up with with your original framework will be used to plan the new project.  Your velocity will look all kinds of weird, and you’ll generally look like you don’t know what you’re doing

Even worse, you might end up having the client accept the more expensive technology, then try to cut costs by trimming stories.  

They’ll use the original numbers you gave them, completely unaware that the original relative complexity is likely to be fairly useless now.

You can try putting a disclaimer in your quote, but that’s fairly certain to be ignored by the client.

My advice, would be:
firstly, don’t give two numbers
secondly, if you have to, do a full re-estimation
thirdly, if you can’t do that, then only give them total numbers, not relative complexity break down. 

Filed under estimation dev software

0 notes &

ruby at_exit can clear return codes

script.rb

at_exit {
  puts "nothing to see here!
  exit
}

puts "error encountered closing system"
exit 1

execute:

ruby script.rb && echo $?

result

error encountered closing system
nothing to see here!
0

So this gets pretty messy when you’re trying to run rspec via rake in a rails project with a bazillion gems around all with their own at_exit hooks (capybara and the web-driver implementations were our problem) It can result in you having failing tests that your CI server don’t report on because it gets a success return code. Our first hack solution was to put our own at_exit block to check for errors and set the return code in spec_helper.rb:

at_exit {
  if check_for_errors
    exit 1
  end
}

Unfortunately some gems are loaded lazily and may append their at_exit hooks after our check and blow away our return code. So our second hack was to nest the at_exit in another at_exit just to try and make sure it’s the last thing that executes:

at_exit {
  at_exit {
    if check_for_errors
      exit 1
    end
  }
}

We’re entering some serious WTF territory there.

It looks odd and it’s still possible that some crazy gem down the path loads a new gem within its at_exit block which causes additional at_exit hooks to be appended. So it’s a fragile solution. Technically we can just keep nesting at_exits in a sort of arms race scenario to try to keep our check at the end of the cleanup code.

The alternative I suppose would be to write the status to a file or other persistance store somewhere and write some specialised code in our rake task to check that store when the tests are finished. I’m not happy with the idea of either solution.

Ultimately it just seems wrong that a call to exit with an error return code gets wiped out by some other cleanup code. I don’t think it quite qualifies as a bug, but it’s definitely a design decision I don’t agree with.

This is another reason why I like Scala more than ruby :P

Filed under ruby dev code gotcha

0 notes &

Why are my watir tests running so slowly?

Today I made a disturbing discovery around the performance of the watir-webdriver framework on firefox.

I inherited some tests that were asserting values on a fairly large html page (12000 lines) within a fairly nested and rigid structure

It looked something like:

div(:class, "outer").ul(:class, "results").li(:class, "result").div(:class, "header").span(:class, "value").text.should_be("7")

Which is apparently the correct and recommended way to query the dom.

When I run this test however, it took 270 seconds to execute!

My belief is that watir is re-building and re-interrogating the dom tree for each element invocation (or causing web-driver to do so, which is equivalent so far as I am concerned)

I migrated this particular test to use capybara and replaced it with the equivalent selector:

find("div.outer ul.results li.result div.header span.value").should have_content("7")

The test execution time dropped to 12 seconds.

Now I wouldn’t expect most tests to experience such a severe difference, the html I was working with is quite a lot of data.

However I think it’s worth knowing that those chained selectors are actually costing you a reasonable amount in test execution time.

Even if it’s only a second or two per test, if you’ve got a high test coverage that’s going to add quite a lot to your test runs. 

Functional tests are already painfully slow, so when you write them, pay attention to doing what you can to speed them up

Filed under ruby dev test gotcha code