Tests Tests Tests

Test Coverage

coverage alarmmanagement

Test Coverage

coverage minicover

Test Coverage

coverage badge

Test Coverage ~0 %

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
public class Useless {
    public static List<(string, int)> GetTop5Word(string txt) {
        return string.IsNullOrEmpty(text)
            ? new List<Tuple<string, int>>()
            : txt.Split(" ")
                 .GroupBy(x => x)
                 .Select(x => (x.Key, x.Count()))
                 .OrderByDescending(x => x.Item2)
                 .Take(5)
                 .ToList();
    }
}
public class UselessTests {
    [Fact]
    public void Coverage0() {
        Assert.Empty(Useless.GetTop5Word(null));
    }
}

Test Coverage 100 %

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
public class Useless
{
    public static double Sin(double i)
    {
        return i;
    }
}

public class UselessTests
{
    [Fact]
    public void Coverage100()
    {
        Assert.Equal(0, Useless.Sin(0));
    }
}

Mehr Tests

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
public class UselessTests
{
    [Fact]
    public void Sin0Test()
    {
        Assert.Equal(0.0, Useless.Sin(0.0));
    }
    [Fact]
    public void Sin30Test()
    {
        Assert.Equal(30.0, Useless.Sin(0.5));
    }
    [Fact]
    public void Sin90Test()
    {
        Assert.Equal(90.0, Useless.Sin(1.0));
    }
}

Data-driven Tests or Raw tests

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
public class UselessTests
{
    [Theory] //[Fact]
    [InlineData( 0.0, 0.0)]
    [InlineData(30.0, 0.5)]
    [InlineData(45.0, 0.70711)]
    [InlineData(60.0, 0.86603)]
    [InlineData(90.0, 1.0)]
    public void SinTest(double input, double expected)
    {
        Assert.Equal(expected, Useless.Sin(input));
    }
}

Attribute Fact Theory

Data-driven Tests or Raw tests

Schon besser, aber...

  • Erlaubt sind nur primitive Typen (string, int, double, etc.)
  • Werte müssen vorberechnet werden
  • Immer noch Stichproben - nicht alle Werte Abgedeckt
  • Test coverage % bleibt groß und ineffektiv

Property-based tests

Was sind Properties?

1: 
2: 
3: 
4: 
5: 
6: 
public class MyProperties {
    public Guid Id { get; set; }
    public string StringProperty { get; set; }
    public int OtherProperty { get; set; }
    public int AnotherProperty { get; set; }
}
Aber Achtung! ...
C# Properties haben damit nichts zu tun 😀

Property

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
var list = List<string> {
    "Alles", "kann", "man", "nicht", "testen"
};

public bool IsOrdered(List<string> list) {
    if (list.Count < 2)
        return true;
    for(var i; i++; i < list.Count - 1)
        if (list[i] > list[i+1])
            return false;
    return true;
}

"Ist die liste sortiert?" - JA/NEIN

Kein Property Test

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
[Fact] // [Theory]
//[InlineData(list,orderedList)]
public void MySortTest()
{
    var input = List<string> {
        "Alles", "kann", "man", "nicht", "testen"
    };
    var expected = list.OrderBy(x => x).ToList();
    var actual = MySort(input);
    Assert.Equal(expected, actual);
}
Nur EINE Stichprobe

Property Test

  • Typ parameter bestimmt den gewünschten Eingabetyp
  • Generator - generiert zufällige Eingabe, konfigurierbar
  • Test - 100 mal, Anzahl konfigurierbar
  • Shrink - minimale Fehlerbedinung finden

Property Test Beispiel

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
using FsCheck;

[Fact]
public void MySortTest() {
    Prop.ForAll<List<string>>(list => {
        var expected = list.OrderBy(x => x).ToList();
        var actual = MySort(list);
        // return (expected == actual);
        // return IsOrdered(actual);
        Assert.True(expected.SequenceEquals(actual));
    }).QuickCheck("MySort");
}

zwei verschiedenen Wege zum Ziel:

Property

1: 
2: 
3: 
4: 
5: 
6: 
7: 
[Fact]
public void JsonSerializerTest() {
    Prop.ForAll<AlarmViewModel>(input =>
        input.ToJson().FromJson<AlarmViewModel>() == input
        // in C# leider Referenzvergleich
    ).QuickCheck("MySort");
}
  • ... wenn keine Vorgabe existiert
    - Z.B. Serializer, Daten-, Format-Converter, etc.
    - Eine Schleife bauen
    - Eingabe und Ausgabe vergleichen

Property

1: 
2: 
3: 
4: 
5: 
6: 
7: 
[Fact]
public void TicketPostTest() {
    Prop.ForAll<TicketViewModel>(ticket => {
        var controller = ...
        Assert.DoesNotThrow(() => controller.Post(ticket));
    }).QuickCheck("MySort");
}

Verhalten abgrenzen - für alle Eingaben keine Exceptions

FsCheck - Testframeworks

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
using FsCheck.XUnit;

[Property] // <--- Property Attribute
public bool TicketPostTest(TicketViewModel ticket) {
    var controller = ...
    Assert.DoesNotThrow(() => controller.Post(ticket));
    return true;
}
  • NUnit
  • XUnit
  • Expecto
  • ...

FsCheck - Testausgabe

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
[Property]
public bool Increment_Twice_Is_The_Same_As_Adding_Two(int x)
{
    return
        (Add(1, Add(1, x)) == Add(x, 2))
        .Classify(x > 10, "Bigger than '10'")
        .Classify(x < 1000, "Smaller than '1000'");
}
    Ok, passed 100 tests.
    63% Smaller than ‚1000‘.
    37% Smaller than ‚1000‘, Bigger than ’10‘.

FsCheck.XUnit output

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
public class Test
{
    private readonly ITestOutputHelper _TestOutputHelper;
    public Test(ITestOutputHelper testOutputHelper)
    {
        _TestOutputHelper = testOutputHelper;
    }

    [Fact]
    public void Test1()
    {
        Prop
            .ForAll(...)
            .VerboseCheckThrowOnFailure(_TestOutputHelper);
    }
}

FsCheck Generator

Strings der gewünschten Länge generieren, z.B. für DB-Column

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
[Property]
public bool SomeProviderTest1()
{
    var str50 = Arb
        .Generate<string>()
        .Where(s => s != null && s.Length <= 50);
    Prop.ForAll(str50, description =>
    {
        var isSet = alarmProvider
            .SetDescription(alarmId, description);
        Assert.True(isSet);
    }).QuickCheckThrowOnFailure();
}

FsCheck Generator

Mehrere Test-Generatoren in einen kombinieren

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
var arbAddress =
    from city in str50
    from street in str100
    select (city, street);

Prop.ForAll(arbAddress, (city, street) =>
{
    var isUpdated = updater.SetAddress(city, street);
    Assert.True(isUpdated);
}).QuickCheckThrowOnFailure();

FsCheck Generator

Eigenen Generator registrieren, z.B. für abgeleitetet Klassen

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
public class TicketBodyGenerator
{
    public static Arbitrary<ITicketBody> TicketBody()
    {
        var genMaintenance = Arb.Generate<Maintenance>().Select(x => (ITicketBody) x);
        var genLimitViolation = Arb.Generate<LimitViolation>().Select(x => (ITicketBody) x);
        var genTimeout = Arb.Generate<Timeout>().Select(x => (ITicketBody) x);
        var gen = Gen.OneOf(genMaintenance, genLimitViolation, genTimeout);
        return Arb.From(gen);
    }
}
...
Arb.Register<TicketBodyGenerator>(); // <-- vor der Benutzung
Prop.ForAll((ITicketBody body) => { ... });

Property Shrink

1: 
2: 
3: 
4: 
5: 
[Property]
public bool StringToLowerMakesEveryCharLower(string input)
{
    return input.ToLower().All(c => Char.IsLower(c));
}

Shrink

Links

Questions?

Donald Knut

Beware of bugs in the above code; I have only proved it correct, not tried it.

Donald Knuth