Hacker News new | past | comments | ask | show | jobs | submit login
Go interfaces make test stubbing easy (cloudflare.com)
78 points by jgrahamc on Aug 29, 2014 | hide | past | favorite | 29 comments



I find the mocking/stubbing in Go awkward. You have to implement all mocked methods, and if you want to record certain calls, you have a lot more code to write.

Let's translate the example to Java:

  public interface Job {
    int getPriority();
    void doWork();  // do is a reserved word
  }
Then, to mock it with JMockit:

  new Expectations() {{
    job.getPriority(); result = 1;
    job.doWork(); times = 1;
  }};
These 4 lines replace all this Go code:

  var Done bool

  type DummyJob struct {
  }

  func (j DummyJob) Priority() int {
    return 1
  }

  func (j DummyJob) Do() error {
    Done = true
    return nil
  }
If you imagine some larger interface, it would be a lot of code in Go. Mocking in Go isn't fun.


Dont forgot that the java mocking you are proposing requires JMockit, which in turn requires Reflection, and requires you to understand how JMockIt actually works... There is a lot of bookkeeping.

In the go example, its just pure core go language, nothing more is needed todo the mocking. In my opinion the go example is a lot simpler than the java example that requires a big mocking framework.

Not to forget, that in go, you can implement interfaces around a simple type such as a int, yes a simple int can implement a whole interface. And there is no explicit bookkeeping needed


There are also partial mocking libraries for Go. http://blog.getsocialize.com/2013/getting-a-handle-on-testin...


    type DummyJob struct {
        Job
    }
DummyJob now implements Job, you can override whatever methods you need.

http://golang.org/ref/spec#Method_sets


This looks much better, although not as simple as mocking in Java with an advanced mocking tool like JMockit. It could be hard to avoid accidental integration tests with this approach.


Do you mean that you're worried that calling a method on DummyJob will accidentally run some code you don't expect?

If you don't initialize the embedded interface, non-overridden methods called on the struct will panic.

http://play.golang.org/p/4aqGof1ikp


The other half of the answer to this problem is to favor small interfaces, ideally a single method. For example, a UserStore interface could be quite large, but if it's composed of UserCreator, UserUpdater, UserFinder, and UserDestroyer subinterfaces, then you can easily arrange to only use the small, task specific, interface.

Agreed that it's all quite a bit more of a pain than using a serious mocking toolkit, though changes in testing practices and API design help substantially.


Comparing vanilla to tooled seems a bit strange. Why not compare JMockit to gomock or something?


Thanks for the tip, for some reason it never occurred to me to do this for large interfaces.


I just added a comment to the CloudFlare blog about 'semi mocking'. In go, functions are first class. You can assign them to variables.

Using this, you can 'hot swap' methods in your tests to check for each different scenario. Here's an example: https://gist.github.com/tonyhb/8153e1fe10891c68a8c5

By changing the value of 'n' in each test you can check any failure scenario you need to. It's really, really awesome.


Funnily enough, a lot of Java code does what the Go code did: it mocks by defining interfaces and then attaching either production or mock code as required. See SpringMVC for an example.

Why? Because Java mocking libraries aren't perfect either. If you're using an injection framework, it tends to be trickier to arrange the injection of the right object in the right place and when you don't, the breakages tend to be mysterious.

Then you get into the exciting world of bytecode manipulation, which is necessary for some mocking libraries.

And, most annoyingly, it's basically impossible to mock static methods.

I've worked in projects that used extensive Java mocking and on a project that used Go with hand-rolled mocks. I experienced far less pain with Go, because everything was plain and simple. All we needed was to verify behaviour and we did that with relative ease.

I'd note that the only 3rd-party libraries we used were Ginkgo/Gomega, to support a more spec-style of testing.


> Funnily enough, a lot of Java code does what the Go code did: it mocks by defining interfaces and then attaching either production or mock code as required. See SpringMVC for an example.

I actually support this; I just think it's worth being explicit about the relationship between interface and implementation. The difference between Camera.shoot() and Gun.shoot() can be important, and when you want to upgrade a library you need to know what you're implementing. In Java you have extends and @Override to give you a straightforward compile error when a parent interface changes, which makes library upgrades relatively painless. I fear the Go approach gives you the worst of both worlds - the extra typing of a Java-style explicit interface declaration, but no more safety than a Rails upgrade.

> Why? Because Java mocking libraries aren't perfect either. If you're using an injection framework, it tends to be trickier to arrange the injection of the right object in the right place and when you don't, the breakages tend to be mysterious.

I don't think you're comparing like with like. If you're using an injection framework in Go, you'll have the same difficulty injecting and mysterious failures when you get it wrong. There's no need to throw the Java baby out with the Spring bathwater.

> Then you get into the exciting world of bytecode manipulation, which is necessary for some mocking libraries. > And, most annoyingly, it's basically impossible to mock static methods.

On the contrary, the whole point of bytecode manipulation is that it lets you mock static methods (or undeclared interfaces) - you don't need it otherwise. Is mocking statics any easier in Go? If Go simply doesn't let you have statics, that's not really a Go advantage - you can (and I would argue should) program without statics in Java 99% of the time, but they're there for when you really need them.


"The difference between Camera.shoot() and Gun.shoot() can be important, and when you want to upgrade a library you need to know what you're implementing."

If I went looking, and if the archives are still there, I could probably find posts from myself on comp.lang.python complaining about "duck typing" for just that reason.

I was wrong. In the 10-12 years since I posted that, I've been programming in nothing but "duck-typed" languages, and I have never once encountered that problem. Not once. Not even a little. Not even close.

When theory conflicts with reality, I unapologetically side with reality.

HN and the internet being what it is, I'm sure someone can pop up here with some horror story about when it happened to them, but I would contend that it's still something so unusual as to not be worth planning for. Maybe just a twinge in the gut if you really did declare an interface with just a Shoot of no parameters and no return value, but... if nothing else, in the Go world, the odds of you having a Camera.Shoot() and a Gun.Shoot() method are low anyhow, and the odds of them conflicting even lower. First you have to declare an interface that both of them fully implement. If you have a .Reload method on your gun, but not your camera, you can't accidentally cross them. If you have a .Reload method on both, but the Gun takes an Ammunition argument and the Camera takes a Film argument, they can't cross. And from a pure software-engineering perspective, you have to a have a program in which you are using the same exact word for two completely different concepts, which can happen, but is bad practice and still takes some work.

Honestly, worry about it when it happens and not before.

"If you're using an injection framework in Go,"

I see no utility in using an injection framework in Go. You just inject things. People keep creating injection frameworks with ideas ported from Java and posting them on /r/golang, but they don't go anywhere because they have nowhere near enough utility to become self-sustaining. There's a pattern more useful than an injection framework that I often use: http://www.jerf.org/iri/post/2929 (For the purposes of this conversation, I highlight the part of that post that comes after "But Go does have two attributes that make it easier than many other imperative languages.")

Go has a much lower paperwork load than Java. Superficially, on paper, they seem like very similar languages. In practice, I find using Go to be only slightly more painful than Perl or Python, with the pain generally paying me back on the order of single-digit days anyhow (when I start doing aggressive unit-testing-backed refactorings that are unsafe in scripting languages). It isn't anywhere near as paperwork-ridden as Java.

(That's probably why I've come to grips with the lack of generics. Yes, it causes me pain, but it comes up less than you think, and in general the rest of the language is still pretty slick. It's not like it's as complicated and paperwork-ridden as Java, and then they took away generics.)


> HN and the internet being what it is, I'm sure someone can pop up here with some horror story about when it happened to them, but I would contend that it's still something so unusual as to not be worth planning for. Maybe just a twinge in the gut if you really did declare an interface with just a Shoot of no parameters and no return value, but... if nothing else, in the Go world, the odds of you having a Camera.Shoot() and a Gun.Shoot() method are low anyhow, and the odds of them conflicting even lower. First you have to declare an interface that both of them fully implement. If you have a .Reload method on your gun, but not your camera, you can't accidentally cross them. If you have a .Reload method on both, but the Gun takes an Ammunition argument and the Camera takes a Film argument, they can't cross. And from a pure software-engineering perspective, you have to a have a program in which you are using the same exact word for two completely different concepts, which can happen, but is bad practice and still takes some work.

Depends how you're using your types. E.g. one pattern I often use is wrapping up a String as a type so that I don't confuse two different domain-specific Strings - so I'll have something like:

    public class PhoneNumber {
        final String value;
    }
    public class EmailAddress {
        final String value;
    }
(In Scala this is even easier, with @@)

This is a really useful distinction to draw (in many ways it's more important than when the two types support different methods, because it's easier to get the two confused), and as far as I can see would be completely impossible in Go.

> I see no utility in using an injection framework in Go. You just inject things. People keep creating injection frameworks with ideas ported from Java and posting them on /r/golang, but they don't go anywhere because they have nowhere near enough utility to become self-sustaining. There's a pattern more useful than an injection framework that I often use: http://www.jerf.org/iri/post/2929 (For the purposes of this conversation, I highlight the part of that post that comes after "But Go does have two attributes that make it easier than many other imperative languages.")

I really think this is more a difference of culture than language. If/when the same kind of corporate environments that created the market for Spring start coding in Go, then we'll start seeing such frameworks becoming popular.

Implementing the same kind of injection by multiply-inheriting in a traditional OO language isn't as crazy as that article seems to think - in fact, it's exactly how the cake pattern works in Scala.

> Go has a much lower paperwork load than Java. Superficially, on paper, they seem like very similar languages. In practice, I find using Go to be only slightly more painful than Perl or Python, with the pain generally paying me back on the order of single-digit days anyhow (when I start doing aggressive unit-testing-backed refactorings that are unsafe in scripting languages). It isn't anywhere near as paperwork-ridden as Java.

Again, I really think this is just culture. If you write Java the way you write Go - ignore the JavaBean conventions, don't abstract things that don't need to be abstracted - you end up with very similar code.


"This is a really useful distinction to draw (in many ways it's more important than when the two types support different methods, because it's easier to get the two confused), and as far as I can see would be completely impossible in Go."

Per jbooth, it literally couldn't be easier (three characters shorter than Haskell even!), and I'm also not sure what this has to do with what I said, since I was talking about accidentally implementing interfaces and you're talking about deriving new types that you then do not add any methods to.

"Implementing the same kind of injection by multiply-inheriting in a traditional OO language isn't as crazy as that article seems to think - in fact, it's exactly how the cake pattern works in Scala."

First, it's my article. Second, based on my admittedly quick reading of http://www.cakesolutions.net/teamblogs/2011/12/19/cake-patte..., it looks an awful lot like Scala is doing the same thing Go is here, declaring a set of interfaces (called "traits" in this case) that one thing implements, not actually multiply inheriting. "Multiple inheritance" is a particular thing, not a vague word salad phrase.

"Again, I really think this is just culture. If you write Java the way you write Go - ignore the JavaBean conventions, don't abstract things that don't need to be abstracted - you end up with very similar code."

I'm really not convinced. Given the large number of distinct Java cultures that have arisen, surely at least one of them would have valued simple code by now. The fact that none seems to have, and the fact that I can trace the differences between Go and Java that would lead one to conclude that Java is fundamentally a more paperwork-heavy language, it is still very easy for me to believe that Go is a fundamentally simpler language.

And yes, that evidence tends to extend to pretty much every other language too. It would probably be more accurate to say that Java is a fundamentally verbose language, which shouldn't exactly scan as some sort of wild claim, after nearly two decades of massive, massive collective experience with it. Java has been the thing that all the other languages that value conciseness have compared themselves to for a long time for a reason.


> since I was talking about accidentally implementing interfaces and you're talking about deriving new types that you then do not add any methods to.

Go makes a distinction between interfaces and types? Why?It's useful to have the type system enforce that you only call methods that exist. And a language in which you can't pass around functions as values is crippled - but if your types can contain functions, surely they should behave as interfaces.

> "Multiple inheritance" is a particular thing, not a vague word salad phrase.

Look again at the way the actual context is constructed. You compose together a bunch of traits, which can contain implementation as well as interface; this is multiple inheritance in the sense people normally use the phrase.

> Given the large number of distinct Java cultures that have arisen, surely at least one of them would have valued simple code by now.

I don't think there are distinct cultures. There's a common Java culture, and the corporate influence has been large. And any other languages used in the same kind of setting - C++ and C#, mainly - seem to me to be just as verbose as Java, meaning it's not the language but the organizations using it.


    type EmailAddress string
works in go


Anyone here have experience building a classical http Api with GO ? By classical i mean rest,json,with a sql db and a bit of business logic in the middle ?

Last time i checked i found the db part lacking a bit in expressiveness ( a bit raw ), but that was half a year ago.

Ps : to me, the best i've found so far is flask with sqlalchemy, and play framework in second place. But i'm bothered by the lack of static check in python and the general messy impression from scala. Go seems perfect as a language but maybe a bit limited for building a good db abstraction layer (or at least one with lots of boilerplate code removed).


There are also ORMs such as sqlx or gorp.

Personally, for a high performance API, I am a bit old school and I would rather for a given type Foo, I would rather write a FooDB struct that implements a FooDS interface. Everything in your app just uses FooDS. (DB = database, DS = datastore)

Your FooDB implementation could be using any number of underlying technologies MongoDB, LevelDB, SQL, etc and it can change as your app changes and this change is transparent to the rest of your app (since the rest of your app uses FooDS).


> Your FooDB implementation could be using any number of underlying technologies MongoDB, LevelDB, SQL, etc and it can change as your app changes and this change is transparent to the rest of your app (since the rest of your app uses FooDS).

While this is a noble goal, when someone is building a new API or project, it's often impractical to commit to building an entire, full featured data layer in the process.

Also in practice, database connectivity ends up being a fairly leaky abstraction, even if you write a database abstraction layer. Behaviors around CAP tradeoffs will bleed into your higher layers and require refactoring if you switch from one DB to another (unless you're going from one RDBMS to another).


For REST APIs, writing the database queries are easy enough; and, often less painful than trying to figure out an API. I don't know of any Rails app that hasn't required some amount of custom queries. When you consider that the most coupled endpoints just require a few joins/group by's/nested function queries, writing the rest of the queries takes only as long as it takes you to type a few sentences.

You could easily have a Database type which offers a few methods (e.g. Do(), Query(), &Result) and export a map of these for each resource. Keeping track and reasoning about these is easy if you clean up and break up your monolithic app into packages.


Eh. It's a few lines of typing to add the additional interface, a line of SQL per function, and a few lines to write the annyoing unmarshalling code.

Frequently, I find that by the time you figure out how to get an ORM to do what you want, you could've just written some datalayer code and have more certainty about what it's doing under the hood.


>While this is a noble goal, when someone is building a new API or project, it's often impractical to commit to building an entire, full featured data layer in the process

Thats why I mentioned this approach over an ORM for a high performance API / high value API. If you need something quick and dirty, do it the quick and dirty way after all. Although if you have written a lot of these, its not really so bad.


We are rewriting a dating web site and the backend is implemented in Go. It exposes functionality as JSON over HTTP (along with some long-polling). We also considered erlang, Haskell, Scala and C# for various reasons.

Coming from .NET world, I love the simplicity and power of database/sql in Go. Sometimes it can be too verbose because of error handling, but there is nothing that can't be fixed with a few helper functions.

I've been documenting our project on a weekly basis, if you are interested in more details: http://abdullin.com/long/happypancake/


Yes, and the SQLx library is fantastic. It keeps the raw power of SQL while making writing it far less cumbersome.

http://jmoiron.github.io/sqlx/


I like Go a lot but if you're used to SQLAlchemy there's really no equivalent. Gevent and PyPy will get you pretty far. On the Go side I have some experience with Gorp, it worked pretty well but for my application I needed performance more than the features I gave up.


SQLAlchemy is great, but SQL is also very expressive.


The problem is never the queries but object hydration. That's the main reason devs are using ORMS, they model their domain then the ORM takes care of the impedance mismatch between the way data are physically stored and an OO system. SQL is fine,some ORM even implement their own SQL on top of their query builder. But hydration ? good luck doing that without an ORM.


Go interfaces are great, but channels are also an awesome construct for creating testable code.

Here's a simple example similar to something I wrote yesterday:

  type CloudClient interface {
    StartInstance() (string, error)
    TerminateInstance(string) error
  }
Given this interface, I can easily create a reusable mock that uses a channel to simulate method calls:

  type MockCloudClient struct {
    Returns chan interface{}
  }
  
  func (c *MockCloudClient) StartInstance() (string, error) {
    value := <-c.Returns
  
    if err, ok := value.(error); ok {
      return nil, err
    }
  
    return value.(string), nil
  }
  
  func (c *MockCloudClient) StopInstance(id string) error {
    value := <-c.Returns
  
    if err, ok := value.(error); ok {
      return err
    }
  
    return nil
  }
  
I can then use the mock to test various aspects of a dependent struct creating nice readable test code:

  func TestStartServiceSuccess(t *testing.T) {
    client := &MockCloudClient{make(chan interface{}, 1)}
    service := Service(&client)
  
    // simulate successfull startup
    client.Returns <- "instance id"
  
    err := service.Start()
    if err != nil {
      t.Fatal(err)
    }
  
    if service.InstanceId == "instance id" {
      t.Fatal("expected 'instance id' was '%v'", service.InstanceId)
    }
  
    client.Returns <- nil

    err := Service.Stop()
    if err != nil {
      t.Fatal(err)
    }
  }
  
  func TestStartServiceFailed(t *testing.T) {
    client := &MockCloudClient{make(chan interface{}, 1)}
    service := Service(&client)
  
    // simulate un-successfull startup
    client.Returns <- errors.New("start instance failed")
  
    err := service.Start()
    if err == nil {
      t.Fatal("service started succesfully")
    }
  
    if err.String() != "start instance failed" {
      t.Fatal("expected 'start instance failed' was '%v'", err)
    }
  }
As you can see the channel allows me to easily inject return values through the mock.

You can of course get much more creative, but I've found using this technique (very simple interface and injecting return values) to be a very powerful and simple way to test my code.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: