JSON Import In .NET

by Anton Fri, July 22 2011 13:37

Bantam Is Quitting Services

We as teamaton were using bantam for all of our todos. At the beginning of this year bantam was bought by ConstantContact, and they announced that bantam will cease services as of July 1. Since we are developing our own todo management tool (see our blog), we decided to push the development and use it instead of bantam.

Of course we wanted to take all of our todos with us. We used bantams export feature which gave us a JSON-file with all our tasks (closed and open ones). So I took on the task to write an JSON import feature into our tool.

Json.NET

After a bit of researching, I found that the library Json.NET would suit our import needs perfectly. Applying the deserialization was pretty straightforward – the documentation helped a lot. Here is the code from the Import controller:

[HttpPost]
public ActionResult Import(HttpPostedFileBase file)
{
    var todos = new List<Todo>();
    if (file != null && file.ContentLength > 0)
    {
        var streamReader = new StreamReader(file.InputStream);
        string text = streamReader.ReadToEnd();
        streamReader.Close();
 
        var bantamTodos = JsonConvert.DeserializeObject<IList<BantamTodo>>(text) as List<BantamTodo>;
        todos = bantamTodos.Select(bantamTodo => bantamTodo.ConvertToTodo()).ToList();
        _todoRepository.SaveImport(todos);
    }
 
    return RedirectToAction("List");
}

It just opens the file, extracts the content as a string, deserializes the string into a list of bantam todos, and then converts these bantam todos into our “normal” todos.

Indirection Via BantamTodo-Class

As you can see, I did not convert the JSON directly into our Todo-class. You can use attributes and the converter class to deserialize JSON into a class of your liking. There are two reasons, why I did not choose to do so: I did not want to load the Todo-class with attributes and converters, and I thought it would be easier to introduce a middle class (BantamTodo), which poses as a container and converter.

I used a nice tool, to take a good look into the original JSON-file: JSON Viewer.

JSON Viewer

With the information about the structure of the JSON file I started implementing via the TDD pattern. Here is my test class, which tests the deserialization of the the bantam todos and the conversion from the class BantamTodo to Todo:

[Test]
public void Should_Import_BantamToDo_FromJson()
{
    var jsonToDo = ArrangeTaskAsJson();
    var bantamToDo = JsonConvert.DeserializeObject<BantamTodo>(jsonToDo);
 
    bantamToDo.Categoy.Should().Be.EqualTo("Organisation");
    bantamToDo.Complete.Should().Be.EqualTo(true);
    bantamToDo.Created_At.Should().Be.EqualTo(new DateTime(2011, 6, 30, 0, 41, 57));
    bantamToDo.Due.Should().Be.EqualTo(new DateTime(2011, 7, 1));
    bantamToDo.Author.Name.Should().Be.EqualTo("Anton");
    bantamToDo.Assigned_To.Name.Should().Be.EqualTo("Oliver");
    bantamToDo.Related_To[0].Name.Should().Be.EqualTo("ToDo-Management Tool");
    bantamToDo.Name.Should().Be.EqualTo("Entwicklung nach Gebieten Personen zuordnen - Verantwortliche, Blogs, etc.");
    bantamToDo.Description.Should().Be.EqualTo("some good description");
    bantamToDo.Flagged.Should().Be.EqualTo(true);
}
 
[Test]
public void Should_Convert_BantamToDo_ToTodo()
{
    var jsonToDo = ArrangeTaskAsJson();
    var bantamToDo = JsonConvert.DeserializeObject<BantamTodo>(jsonToDo);
    var todo = bantamToDo.ConvertToTodo();
 
    todo.Status.Should().Be.EqualTo(bantamToDo.Complete ? Status.Closed : Status.Open);
    todo.Description.Should().Contain(bantamToDo.Name);
    todo.Description.Should().Contain(bantamToDo.Description);
    todo.Tags.Select(t => t.Name).Should().Contain(bantamToDo.Categoy);
    foreach (var bantamProject in bantamToDo.Related_To)
        todo.Tags.Select(t => t.Name).Should().Contain(bantamProject.Name);
    todo.DateCreated.Should().Be.EqualTo(bantamToDo.Created_At);
    todo.DateCompleted.Value.Date.Should().Be.EqualTo(bantamToDo.Due);
    todo.DateDue.Should().Be.EqualTo(bantamToDo.Due);
    todo.Creator.Name.Should().Be.EqualTo(bantamToDo.Author.Name);
    todo.Assignee.Name.Should().Be.EqualTo(bantamToDo.Assigned_To.Name);
    todo.Priority.Value.Should().Be.EqualTo(bantamToDo.Flagged ? 2 : 0);
}

The implementation was pretty straightforward. Since it was my first time working with MVC, and also my first time working with JSON, it took me some time. All in all – research, export and meetings included – it took me about 12 hours.

If you have any suggestions as to improvement I would appreciate them. If you are trying to import JSON into .NET yourself, I hope that this article helps.

Tags:

.NET | C# | Software development | Test Driven Development (TDD) | JSON

Mocking revisited – mocking a method and passing a parameter using FakeItEasy and Moq

by Oliver Tue, June 21 2011 23:35

Finally, I’ve started mocking things in our kind of legacy project while building new functionality. Now, I wanted to replace some DAL service with a mock that would would just serve me some results instead of ramping up NHibernate, going to the DB and assembling all the entities for this test. At first I tried to use FakeItEasy as I liked the syntax more than that of Moq:

  1: var _geoObjectServiceMock = A.Fake<IGeoObjectService>();
  2: var results = new List<int>{ 1, 2, 3, ... };
  3: A.CallTo(() => _geoObjectServiceMock.GetSearchResultIds(A<GeoObjectSearchDesc>.Ignored))
  4:     .Returns(results);

This was easy. Just fix up the result you want the method to return and set it up. Well – I needed more. I needed to return my results list dependent on the GeoObjectSearchDesc parameter passed to the method. In short: I did not find a way to do this. I searched Google and Stackoverflow but to no avail. What I did find was an example using Moq showing exactly what I wanted to do. From Moq’s QuickStart wiki page:

  1: // access invocation arguments when returning a value
  2: mock.Setup(x => x.DoSomething(It.IsAny<string>()))
  3:     .Returns((string s) => s.ToLower());
  4: // Multiple parameters overloads available

So I quickly installed Moq using NuGet, e.g. from the Package Manager Console:

  1: Install-Package Moq -project Tests

Now I have something like this which is exactly what I was looking to do:

  1: var _geoObjectServiceMock = new Mock<IGeoObjectService>();
  2: var results = new List<int>{ 1, 2, 3, ... };
  3: _geoObjectServiceMock.Setup(svc => svc.GetSearchResultIds(It.IsAny<GeoObjectSearchDesc>()))
  4:                      .Returns<GeoObjectSearchDesc>(gosd => results.Take(gosd.PageSize).ToList());

That’s what I expected – to get control over the method parameter(s) inside the mock. Great! Thank you Moq.

Happy Coding,

Oliver

Tags:

.NET | C# | Test Driven Development (TDD)

Mocking for the first time – a short overview of a few .NET mocking frameworks

by Oliver Fri, March 25 2011 15:50

In my recent post Testing: trying to get it right I mentioned that a lot of our tests are of the dirty hybrid kind, somewhere between real Unit tests and real Integration tests. Focusing on the Unit test part, we’re looking into using a mocking framework right now to change the way we write tests – most of all to decouple the different components that we use in the application under test.

Wanting to use the fresh and hype NuGet package manager to install the mocking frameworks, I chose among the ones that were both available there and also looked promising:

Really, it could not have been easier to get all these libraries into the project than using NuGet!

Sample code

So I went ahead and wrote a short and simple test just to get a feel of the syntax they offer. At first I wanted to mock a simple Repository using its interface. Here is what I ended up with:

  • Telerik JustMock: using syntax from their quick start manual
        1: public void MockRepositoryInterfaceTest()
      
        2: {
      
        3: 	// Arrange
      
        4: 	var repo = Mock.Create<ITodoRepository>();
      
        5: 	Mock.Arrange(() => repo.Todos)
      
        6: 		.Returns(new List<Todo> {new Todo {Description = "my todo"}}.AsQueryable);
      
        7: 
      
        8: 	// Act + Assert
      
        9: 	repo.Todos.ToList().Count.Should().Be.EqualTo(1);
      
       10: }
  • Moq: just visit the project homepage
        1: public void MockRepositoryInterfaceTest()
      
        2: {
      
        3: 	// Arrange
      
        4: 	var repo = new Mock<ITodoRepository>();
      
        5: 	repo.SetupGet(rep => rep.Todos)
      
        6: 		.Returns(() => new List<Todo> {new Todo {Description = "my todo"}}.AsQueryable());
      
        7: 
      
        8: 	// Act + Assert
      
        9: 	repo.Object.Todos.ToList().Count.Should().Be.EqualTo(1);
      
       10: }
  • FakeItEasy: just visit the project homepage
        1: public void MockRepositoryInterfaceTest()
      
        2: {
      
        3: 	// Arrange
      
        4: 	var repo = A.Fake<ITodoRepository>();
      
        5: 	A.CallTo(() => repo.Todos)
      
        6: 		.Returns(new List<Todo> {new Todo {Description = "my todo"}}.AsQueryable());
      
        7: 
      
        8: 	// Act + Assert
      
        9: 	repo.Todos.ToList().Count.Should().Be.EqualTo(1);
      
       10: }

    In the first test-ride FakeItEasy and JustMock look pretty much identical, whereas the syntax Moq offers is a bit awkward with the SetupGet() method name and the need to call repo.Object to get the instance. I hope to examinate further differences in use as the project moves on.

    Mocking concrete classes

    Since we’re working with a large application that still has a lot of services and classes not implementing any interface I also wanted to make sure we’ll be able to mock concrete types. Well, this didn’t go so well:

    different-mocking-frameworks-all-fail-on-concrete-classes

    JustMock and FakeItEasy simply returned an instance of the concrete class I gave them, Moq complains that it can’t override the Todos member. So I added the virtual modifier to it and the test is now green. Still, I got the impression that I was trying to do something that I shouldn’t. The following blog post motivates further not to mock concrete classes: Test Smell: Mocking concrete classes. So I guess introducing interfaces as a kind of contract between classes is the way to go, but in the meantime and where we can’t avoid mocking concrete types we’ll be left using Moq.

  • That’s it for now, happy coding!

    Oliver

    Testing: trying to get it right

    by Oliver Wed, February 09 2011 10:54

    Read a great post on Steve Sanderson’s blog with the promising title Writing Great Unit Tests – and it is definitely worth reading. He mentions another post with the title Integration Testing Your ASP.NET MVC Application which I also recommend. One of the eye openersfor me was this quote in his post:

    TDD is a design process, not a testing process”. Let me elaborate: “TDD is a robust way of designing software components (“units”) interactively so that their behaviour is specified through unit tests.”

    I must admit that I haven’t read much yet about TDD – but we’ve been writing tests for quite some time now. Unfortunately, most of them probably fall into the Dirty Hybrids category that Sanderson sees between two good ends, one being True Unit Tests, the other being Integration Tests. I allow myself to add his illustration here:

    image-thumb1[1]

    So it looks like our goal should be to write tests in either of the outer categories and slowly but surely get rid of the time consuming, easy-to-break hybrid tests.

    One problem that a lot of people writing web applications are confronted with at some point, is testing the whole application stack from browser action over server reaction to browser result. We’ve put some effort into abstracting away the HttpContext class to use the abstraction in both our frontend and in tests, but it falls short of being a worthy replacement for the real HttpRequest and HttpResponse classes. With all that works we’re missing a possibility to use our abstraction in a third party URL Rewriting engine, so the requests we are testing never get processed by it. We have tests for the rules that are applied by the engine, but for more sophisticated setups this simply is not enough. Thanks to a link on Steve Sanderson’s blog post on integration testing ASP.NET MVC applications I stumbled upon Phil Haack’s HttpSimulator – and it looks just like the piece in the puzzle we’ve been missing for all that time. (I have no idea how we didn’t find that earlier.)

    Another thing I’m new to is kata. Kata is a Japanese word describing detailed choreographed patterns of movements practiced either solo or in pairs. A code kata is an exercise in programming which helps hone your skills through practice and repetition. At first, it might sound weird to practice and repeat the same exercise over and over again. But then again, if we think of musicians or sportsmen it’s not hard to see that they become great at what they do only by practicing. A lot. And they practice the same move (let’s just call it that) over and over again. The idea behind the code kata is that programmers should do the same. This is what Dave Thomas, one of the authors of The Pragmatic Programmer, also promotes on his blog.

    I stumbled upon a very interesting kata in a blog post by Robert Martin about test driven development, which deals with the evolution of tests and poses the assumption that there might be a kind of priority for test code transformation that will lead to good code: The Transformation Priority Premise. The kata he mentions further down is the word wrap kata. The post is rather long so count at least 15 min to read it. For me it was well worth it. The solution to a very simple problem, at least a problem that’s easy to explain, can be quite challenging – and the post shows hands on how writing good tests can help you find a good solution faster. It showed me (again) that writing good tests lets you write good code. Just like the quote above states: TDD is a software development process.

    Happy coding – through testing ;-)

    Oliver

    Tags:

    Test Driven Development (TDD) | Testen | Software development

    TDD != Unit-Test getrieben

    by robert Sat, March 27 2010 20:08

    Wer Testgetriebene-Entwicklung betreibt, muss nicht zwingend auch Unit-Test getrieben entwickeln.

    Testgetrieben, sowohl in Greenfield also auch in Brownfield Projekten, ist für mich zunehmend Integrations oder Programmer-Test getrieben. Tatsächliche Unit-Tests (Testen von ageschlossenen Modulen) geraten zunehmend und nach Anwendungscharakter immer mehr in die Mindertheit.

    Tags:

    Test Driven Development (TDD)

    NHibernate: Fetch Join + EntityCache

    by Oliver Fri, October 16 2009 13:28

    In unserer Test-basierten Entwicklung hat sich die Verwendung von Setup-Klassen für unsere Entities etabliert. Unsere Tests sehen folgendem sehr ähnlich:

    [Test]
    public void FetchJoinTest()
    {
        var cat = _categorySetup.GetPersisted();
        var pitch = _pitchSetup.GetPersisted(p => { p.Categories.Add(cat); return p; });
        var reserv1 = _reservationSetup.GetPersisted(r =>
                                                         {
                                                             r.Pitch = pitch;
                                                             r.DateFrom = new DateTime(2009, 10, 31);
                                                             r.DateTo = new DateTime(2009, 11, 13);
                                                             return r;
                                                         });
     
        pitch = _pitchService.GetById(pitch.Id);
        Assert.That(pitch.Reservations.Count, EqualTo(2));
    }
     

    Leider funktioniert der Test so nicht - er schmeißt eine AssertionException, da pitch.Reservations keine Elemente enthält.

    Warum nicht???

    Nach langem hin und her, bin ich mir endlich sicher, es verstanden zu haben: es liegt an NHibernate's Entity-Cache! Da das Pitch-Objekt ja zum Zeitpunkt der Abfrage

     
    
        pitch = _pitchService.GetById(pitch.Id);
    
     
    

    noch in der aktuellen Session geladen ist (was NHibernate meines Wissens nach anhand der ID feststellt), wird kein neues Objekt vom Typ Pitch instanziiert, sondern das schon existierende, pitch, zurückgegeben. Dieses enthielt aber noch keine Reservierungen zum Zeitpunkt der Persistierung - und so, wie es aussieht, werden diese auch nicht nachträglich geladen (für ein Objekt, das schon in der Session lebt).

    Die Lösung ist jetzt gar nicht mehr so schwer: wir schmeißen das vorhandene Objekt einfach weg! Dafür gibt es Session.Evict():

    [Test]
    public void FetchJoinTest()
    {
        var cat = _categorySetup.GetPersisted();
        var pitch = _pitchSetup.GetPersisted(p => { p.Categories.Add(cat); return p; });
        var reserv1 = _reservationSetup.GetPersisted(r =>
                                                         {
                                                             r.Pitch = pitch;
                                                             r.DateFrom = new DateTime(2009, 10, 31);
                                                             r.DateTo = new DateTime(2009, 11, 13);
                                                             return r;
                                                         });
     
        _nHibernateHelper.Session.Flush();
    
        _nHibernateHelper.Session.Evict(pitch);
    
     
        pitch = _pitchService.GetById(pitch.Id);
    
        Assert.That(pitch.Reservations.Count, EqualTo(2));
    
    }
    

    Jetzt läuft der Test mit Erfolg durch. Happy Coding!

    Tags:

    NHibernate | Test Driven Development (TDD)

    Cache, Autofac und Tests

    by Oliver Wed, June 10 2009 10:33

    Gestern ging wieder etwas Zeit drauf, als ich unsere Tests ganz machen wollte und die Tests für das UrlRewriting partout nicht in der Testsuite durchlaufen wollten. Einzeln ausgeführt funktionierte hingegen alles wunderbar.

    Dieses Phänomen tritt immer mal wieder auf, und es liegt (natürlich ;-)) an den unterschiedlichen Ausgangssituationen, in denen der betroffene Test startet.

    Ohne große Umschweife hier also drei potenzielle Fallen:

    • Der Cache ist gefüllt oder unsere RepositoryDb-Basisklasse hat nach GetAll() eine Liste aller Elemente zwischengespeichert (siehe auch 2.)
    • Einige Services haben einen Zustand und dieser ist anders, als im Test angenommen --> Autofac-Container recyclen
    • Die DB wird durch Custom-SQL manipuliert (bspw. zur Vorbereitung eines Tests) und alle Businesslogik wird umgangen.

    Wenn also Fehler mit Tests auftreten, bitte diese drei Punkte überprüfen. Für Punkt eins und zwei gibt es in der BaseTest-Klasse die Methode RecycleServiceContainerAndClearCache().

     

    Tags:

    Test Driven Development (TDD) | Tips & Tricks

    TDD: Testen ob eine Cookie gesetzt wurde

    by robert Thu, May 14 2009 16:14

    Für Webentwickler ist das Auswerten und Setzen von Cookies Business-Logik und Business Logik sollte Testgetrieben entwickelt werden.

    Hier ein Beispiel Unit-Test:

    Response _response { get { return _httpCurrent.Response as ResponseNoWeb; } }
    RequestNoWeb _request { get { return _httpCurrent.Request as RequestNoWeb; } }
     
    [Test]
    public void SetCookie()
    {
        _request.Url = new Uri("http://de.camping.info");
     
        _uiLanguageService.SetLanguageCookie("de");
        
        Assert.That(_response.CookiesCreated[0].Value, Is.EqualTo("de"));
        Assert.That(_response.CookiesCreated[0].Name, Is.EqualTo("Lang"));
        Assert.That(_response.CookiesCreated[0].Domain, Is.EqualTo(".camping.info"));
    }
     
     
    Ermöglicht wird das über die Abstrakton von HttpRequest und HttpResponse. (Die Entwicklung wurde gestern begonnen, der Quelltext kapselt daher nur das Grundlegenste.)
     
    Erfolgt die Anwendung nicht im Webkontext verwenden wir eine Implementierung für den lokalen Gebrauch:
     
    namespace SpeakFriend.Utilities.Web
    {
        public class ResponseNoWeb : Response, IResponse
        {
            public void Redirect(string url)
            {
                _redirections.Add(url);
            }
     
            public void Redirect(string url, bool endResponse)
            {
                _redirections.Add(url);
            }
     
            public void SetCookie(HttpCookie cookie)
            {
                _cookiesCreated.Add(cookie);
            }
        }
    }

    …rudimentär, aber wirksam.

    Tags:

    Web Applikationen | Test Driven Development (TDD)