FsSpec.FsCheck 0.1.0-alpha3

FsSpec

What is FsSpec and why would you use it?

Short

FsSpec represents value constraints as data to reuse one constraint declaration for validation, data generation, error explanation, and more.

It also makes for a concise and consistent Type-Driven approach

open FsSpec
type InventoryCount = private InventoryCount of int
module InventoryCount = 
    let spec = Spec.all [Spec.min 0; Spec.max 1000]
    let tryCreate n =
      Spec.validate spec n 
      |> Result.map InventoryCount

// Generate data
let inventoryAmounts = Gen.fromSpec InventoryCount.spec |> Gen.sample 0 10

Longer

Type-Driven and/or Domain-Driven systems commonly model data types with constraints. For example,

• an string that represents an email or phone number (must match format)
• an inventory amount between 0 and 1000
• Birthdates (can't be in the future)

We centralize these constraints by wrapping them in a type, such as

type PhoneNumber = private PhoneNumber of string
module PhoneNumber = 
    let tryCreate str =
      if (Regex(@"\d{3}-\d{4}-\d{4}").IsMatch(str))
      then Some (PhoneNumber str)
      else None 

This is great. It prevents defensive programming from leaking around the system and clearly encodes expectations on data. It avoids the downsides of primitive obsession.

However, we're missing out on some power. We're encoding constraints in a way that only gives us pass/fail validation. We have to duplicate constraint information if we want to explain a failed value, generate data, or similar actions.

FsSpec represents these constraints as data so that our programs can understand the constraints on a value.

let inventorySpec = Spec.all [Spec.min 0; Spec.max 1000]

// Validation
Spec.isValid inventorySpec 20

// Explanation: understand what constraints failed (as a data structure)
Spec.explain inventorySpec -1

// Validation Messages
Spec.explain inventorySpec -1 |> Formatters.prefix_allresults // returns: "-1 failed with: and [min 0 (FAIL); max 1000 (OK)]"

// Data Generation (with FsCheck)
Gen.fromSpec inventorySpec |> Gen.sample 0 10  // returns 10 values between 0 and 1000

There are also other possibilities FsSpec doesn't have built-in. For example,

• Comparing specifications (i.e. is one a more constrained version of the other)
• Serialize and interpret constraints for use in different UI technologies
• Automatic generator registration with property testing libraries (e.g. FsCheck)

Basic Value Type using FsSpec

It's still a good idea to create value types for constrained values. Here's how you might do it with FsSpec

open FsSpec
type InventoryCount = private InventoryCount of int
module InventoryCount = 
    let spec = Spec.all [Spec.min 0; Spec.max 1000]
    let tryCreate n =
      Spec.validate spec n 
      |> Result.map InventoryCount

Supported Constraints

• Spec.all spec-list: Logical and. Requires all sub-specs to pass
• Spec.any spec-list: Logical or. Requires at least one sub-spec to pass
• Spec.min min: Minimum value, inclusive. Works for any IComparable<'a>
• Spec.max max: Maximum value, inclusive. Works for any IComparable<'a>
• Spec.regex pattern: String must match the given regex pattern. Only works for strings.
• Spec.predicate label pred: Any predicate ('a -> bool) and a explanation/label
• Spec.minLength min: set a minimum length for a string or any IEnumerable derivative
• Spec.maxLength max: set a maximum length for a string or any IEnumerable derivative

Generation Limitations

Data generation can't be done efficiently for all specifications. The library recognizes special cases and filters a standard generator for the base type for everything else.

The library understands most numeric ranges, date ranges, regular expressions, and logical and/or scenarios. Custom scenarios for other IComparable types would be easy to add, if you encounter a type that isn't supported.

Collection length constraints currently support IEnumerable<T>, lists, arrays, and readonly lists and collections. Dictionaries, sets, and other collections are not yet supported but should not be difficult to add.

However, predicates have limited generation support. For example, this tightly restrictive predicates may fail to generate values.

let spec = Spec.predicate "predicate min/max" (fun i -> 0 < i && i < 5)

The above case will probably not generate any values. It is filtering a list of randomly generated integers, and it is unlikely many of them will be between 0 and 5. FsSpec can't understand the intent of the predicate to create a smarter generator.

Impossible specs (like all [min 10; max 5]), also cannot produce generators. The library tries to catch impossible specs and thrown an error instead of returning a bad generator.

Complex / Composed Types

FsSpec doesn't currently support composed types like tuples, records, unions, and objects.

The idea is that these types should enforce their expectations through the types they compose. Scott Wlaschin gives a great example as part of his designing with types series.

A short sample here.

Sum types (i.e. unions) represent "OR". Any valid value for any of their cases should be a valid union value. The cases themselves should be of types that enforces any necessary assumptions

type Contact = 
  | Phone of PhoneNumber
  | Email of Email

Product types (records, tuples, objects) should represent "AND". They expect their members to filled. If a product type doesn't require all of it's members, the members that are not required should be made Options.

type Person = {
  // each field enforces it's own constraints
  Name: FullName 
  Phone: PhoneNumber option // use option for non-required fields
  Email: Email option
}

Cases with rules involving multiple members should be refactored so the types enforce the expectation. A common example is requiring a primary contact method, but allowing others.

type Contact = 
  | Phone of PhoneNumber
  | Email of Email

type Person = {
  Name: FullName 
  PrimaryContactInfo: Contact
  OtherContactInfo: Contact list
}

See Designing with Types (free blog series) or the fantastic Domain Modeling Made Functional (book) for more detailed examples.

Roadmap

This library is early in development. The goal is to get feedback and test the library in real applications before adding too many features.

The next step would most likely be additional constraint types

• Not spec: Negate any specification.
  • This is easy to add for validation, but makes normalization for inferring generators more complex. It should be do-able, but I have to consider negations of specs (i.e. max becomes min, regex becomes ???) and how that would impact other features like explanation
• Exact value spec: specify a finite list of allowed values

Other lines of inquiry include

• Improve customization
  • Potentially allow meta separate from predicates so a category of custom constraints can leverage case specific info (e.g. if max were implemented as custom, making the max value accessible to custom formatters, comparisons, generators, etc)
  • Explore how users most often need to extend or modify existing functionality.
    • add formatting for their custom constraint?
    • mapping custom errors? / interpreting error scenarios
• Identifying base set of constraints that should be built into the library

Project Status

The most foundational features (validation, generation, explanation) are implemented and tested. The library should be reliable, but the public API is subject to change based on feedback.

The main goal right now is to gather feedback, validate usefulness, and determine next steps, if any.

Inspiration

This library borrows inspiriation from many sources

• Clojure.spec
• Specification Pattern by Eric Evans and Martin Fowler
• Domain Driven Design
• Type-driven Development
  • Designing with Types by Scott Wlaschin
  • Mark Seemann

Original Experiments

I previously looked into adding constraints as a more integrated part of the F# type system. Those experiments failed, but are still available to explore.

If you want such a type system, you might checkout F*, Idris, or Dafny.