Alexandre Moine’s blog

Practical deforestation for fixed-point structures – introducing tungsten

2019-07-26T00:00:00+02:00

This post introduces tungsten, a minimalist library that provides deforestation for free to any structure expressed as a fixed-point. It is proved using free theorems and tested using inspection-testing.

Introduction

I worked earlier this year on bringing deforestation specifically to algebraic graphs, recounted in this past blog post. The whole process was basically copy-pasting things from the GHC.Base module that works on lists and adapting them to work on graphs.

As my first-year professor once told me:

Never copy-paste code. If you copy-paste a line of code, you are either introducing a bug or missing an opportunity to generalize your program.

So, as I was not introducing a bug, it seems that I was missing an opportunity to generalize these rules. The theory already showed that this was possible but there was no (known to me) usable implementation. As the title suggests, I came to writing tungsten, a library (not yet on Hackage) that generalizes the "foldr/build" rule to every structure expressed as a fixed-point.

This post will introduce (quickly) the three needed concepts (namely deforestation, fixed-point structures, and recursion schemes) and how to cook them to obtain cool results. If you are in a hurry and familiar with these concepts, jump to the core section.

Deforestation

The concept of deforestation was introduced by Wadler in his paper “Deforestation: transforming programs to eliminate trees”, in Theoretical Computer Science 73, 2 (1990), 231 – 248.

It is the idea that, when working with “good” functions on recursive structures, one can often optimize their composition to avoid building intermediate structures.

A typical example on lists is the expression:

map g (map f xs)

map f will destruct the original list and produce a new one, which will be destructed right away by the outer map. Knowing the definition of map, it is easy to see that the above expression is equivalent to a more efficient one:

map (g . f) xs

that go through the list only once.

Deforestation is implemented in GHC for functions working on lists using the rewrite rules system (that is, the above optimization is made automatically by the compiler; see this previous blog post).

However, the rewriting operation we made in this example do not depends on the list structure: we just used the fact that we knew how the list produced by map is made. This concept can be simply generalized to any recursive structure (as theoretically explained by Johann in “A Generalization of Short-Cut Fusion and Its Correctness Proof”, Higher-Order and Symbolic Computation (2002) 15: 273).

Currently, if one wants to add deforestation to another structure (such as algebraic graphs) then he needs to add a new set of rules. This is boring, hard to maintain and complicated for a neophyte. The idea is instead to generalize the "foldr/build" rule to catch all these cases.

Firstly, we need a way to think about recursive structures generically: this is made using fixed-points.

The Fix operator

A useful way to think about recursive structures is using fixed-points: they are really often used in mathematics and can be used for types. All the next section was made after the already mentioned recursion-schemes package.

Let me jump right into the subject: the fixed-point operator in Haskell can be defined as:

newtype Fix f = Fix (f (Fix f))

You can find this definition in the Tungsten.Fix module.

What is this strange thing?

The constructor Fix is of type f (Fix f) -> Fix f: it basically “hides” a level of application of the base type. You can think Fix f as a type f applied to itself indefinitely (this is exactly the usual mathematical definition of a fixed-point).

How the hell am I using this?

The simplest thing here is to take an example (everything here can be found in the Tungsten.Structure.List module).

Linked lists can be defined as:

-- The factored-out recursive type for lists.
data ListF a b =
    NilF
  | ConsF a b
  deriving (Eq, Show, Functor)

-- Linked lists as a fixed-point.
type List a = Fix (ListF a)

How is it working? The ListF data represent the two constructors of lists: either the empty one (NilF) or an element and something else (ConsF), if you replace “something else” by another list, you have a linked list.

For example

lst :: ListF Int (ListF Int (ListF Int (ListF Int b)))
lst = ConsF 1 (ConsF 2 (ConsF 3 NilF)) -- ~= [1,2,3]

So, all lists are in the fixed-point of ListF, that is type List a = Fix (ListF a).

Then, we can define:

-- The empty list. Similar to 'Prelude.[]' for Prelude lists.
nil :: List a
nil = Fix NilF

-- The cons operator. Similar to 'Prelude.(:)' for Prelude lists.
cons :: a -> List a -> List a
cons x xs = Fix (ConsF x xs)

lst :: List Int
lst = cons 1 (cons 2 (cons 3 nil)) -- ~= [1,2,3]

Note that being a newtype, Fix is optimized away at compile time: there will be no Fix references in the code produced by the simplifier. This is due to the great work of Breitner et al. described in “Safe Zero-cost Coercions for Haskell” 2014. SIGPLAN Not. 49, 9 (Aug. 2014), 189–202.

Recursion-schemes

Deforestation is about destruction. Consequently, the next question is: how do we destruct fixed-point structures?

It is cata-strophic

foldr is a pretty general pattern (who didn’t write a fold_tree or a fold_my_custom_struct ?). It is so general that there is a whole theory behind this process: recursion-schemes. They were popularized by Meijer et al. in their famous “Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire”, in Hughes J. (eds) Functional Programming Languages and Computer Architecture. FPCA 1991.

I am not going to explore them deeply, there is plenty of well-made blog posts about them and a beautiful library called, well, recursion-schemes.

Particularly, functions that “destruct recursively” (such as foldr for lists) are called catamorphisms. Having this vocabulary, the idea of deforestation can be (roughly) expressed as: if a structure is built recursively and after destructed by a catamorphism, then the intermediate structure does not need to be built.

Catamorphisms for fixed-point structures

Catamorphisms for fixed-point types can be expressed using cata:

cata :: Functor f => (f a -> a) -> Fix f -> a

It is simple to use. If you have a (non recursive) structure (of type f, for example ListF u v) and you want to destruct its fixed-point (Fix f, for example List u) to produce something of type a, you only have to give a function that performs one step of the recursion, that is a function of type f a -> a (for lists, that is a function of type ListF u a -> a). One can think cata go as just a function replacing every occurrence of Fix by go in a fixed-point structure.

Note that the Functor instance is needed to perform recursion since we only know that the structure is a fixed-point, but we don’t have access to its constructors.

An example

As an example of a catamorphism, let us write the function elem :: Eq a => a -> List a -> Bool:

elem :: Eq a => a -> List a -> Bool
elem e = cata go
  where
    -- go :: ListF a Bool -> Bool
    go NilF = False
    go (ConsF a b) = a == e || b

The rule(s)

(Disclaimer: I have removed all the INLINE pragmas and phase control to simplify the reading. Don’t hesitate to have a look at the code to see the details.)

Ok, now, the cool (and new) part.

The main idea is that one can abstract the build of a fixed-point structure with respect to Fix (just like the build function of lists abstract the two lists constructors, see this previous blog post). Then, because cata go is just replacing every occurrence of Fix by go in the structure, the intermediate structure can be avoided. This is done by buildR, which can be defined as:

-- Type of arguments of 'buildR'.
type Cata f = forall a. (f a -> a) -> a

-- Defining a function in terms of 'buildR' allows fusion if
-- this function is composed later with 'cata'.
buildR :: Cata f -> Fix f
buildR g = g Fix

The forall guarantees that the rule is valid (see here).

Because buildR abstracted Fix, we can then replace Fix by a function go if the structure is destructed right away by cata go. This is exactly the purpose of the "cata/buildR" rule:

{-# RULES
"cata/buildR" forall (go :: t a -> a) (g :: Cata t).
  cata go (buildR g) = g go
 #-}

With this single rule, deforestation is provided to every fixed-point structures.

Examples

The "foldr/map" optimization is just a particular case of "cata/buildR" (that is, for example, foldr f z . map g will be optimized in foldr (f . g) z).

Here are the definitions (note that foldr is just cata and map is defined in terms of buildR):

foldr :: (a -> b -> b) -> b -> List a -> b
foldr f z = cata foldr_go
  where
    foldr_go NilF = z
    foldr_go (ConsF a b) = f a b

map :: (a -> b) -> List a -> List b
map g xs = buildR mapBody
  where
    mapBody c =
      let map_go x =
        case x of
          NilF -> c NilF
          ConsF a b -> c (ConsF (g a) b)
      in cata map_go xs

Then we have:

   foldr f z (map g xs)
== cata foldr_go (buildR mapBody) -- Inlining of `foldr` and `map`
== mapBody foldr_go               -- The "cata/buildR" rule
== let map_go x =                 -- Inlining of `mapBody`
     case x of
       NilF -> foldr_go NilF
       ConsF a b -> foldr_go (ConsF (g a) b)
    in cata map_go xs
== let map_go x =                 -- Inlining of `foldr_go`
     case x of
       NilF -> z
       ConsF a b -> f (g a) b
    in cata map_go xs
== foldr (f . g) z xs             -- Identification

Another example

Ok this is cool, but we already had all of this for classical lists (with the "foldr/build" rule)! Here is an example with binary trees (you can find definitions in the Tungsten.Structure.Tree module).

Ready? Let us work with bind for binary trees.

-- The "factored-out" recursive type for binary trees.
data TreeF a b =
    EmptyF
  | LeafF a
  | NodeF b b
  deriving (Eq, Show, Functor)

-- Binary trees expressed as a fixed-point.
type Tree a = Fix (TreeF a)

-- @bind@ for trees, defined in terms of 'buildR' and 'cata'.
-- Good consumer and good producer.
bind :: Tree a -> (a -> Tree b) -> Tree b
bind t f = buildR $ \fix' ->
  let bind_go x =
    case x of
      EmptyF -> fix' EmptyF
      LeafF a -> cata fix' $ f a
      NodeF a b -> fix' $ NodeF a b)
  in cata bind_go t

bind might seems strange, but if you replace fix' by Fix, all is normal (remember, cata fix == id).

Then, the composition of several bind will be optimized using its associativity without anything to do:

dBind, dBindR :: (b -> Tree c) -> (a -> Tree b) -> Tree a -> Tree c
dBind  g f t = bind (bind t f) g
dBindR g f t = bind t (\x -> bind (f x) g)

With the cata/buildR rule, dBind will produce the same Core code as dBindR! dBindR is indeed faster than dBind since the inner bind has direct access to the result of f without going through the original structure.

Proof

Is this optimization valid? Is it proved?

The answer is yes. You can find a more-or-less formal proof in the tungsten repository. Warning, seq and undefined are not good friends here, and they can break the rule (as for "foldr/build").

Another rule

Now if there is no rewriting? All expressions made with buildR might be rewritten to cata Fix x which corresponds to a totally useless destruction/reconstruction of the structure.

That is why Tungsten.Fix also provides another rule:

{-# RULE
-- We cannot target `Fix` since it will be optimized away
"cata/id"
  cata coerce = id
 #-}

The only difficult step is to see that, because Fix is a newtype, the only way to target it is using coerce (from Data.Coerce).

Tests

All of this seems theoretically ok, but does the optimization actually occurs? To test that rules are correctly firing, I use the great inspection-testing package which allows comparing GHC’s Core code of two expressions. All examples presented in this post (and many more) are tested this way (you can see all tests here).

Conclusion

This was pretty fun to write.

Fortunately, there are many paths to explore:

The fusion for paramorphisms (aka. generalized catamorphisms) needs more work to be usable.
For the rules to fire, we need to INLINE pretty much everything. It may increase the size of compiled code. Maybe there is a way to avoid this.
There used to be a foldr2 rule for functions that perform recursion on two lists simultaneously. Is it possible to write cata2 and optimize its composition with buildR?

Thanks

Thanks to Andrey Mokhov who suggested the idea and proofread this article.

Learn-ocaml code classification

2019-06-24T00:00:00+02:00

Introduction

I am currently doing a summer internship at IRIF under the supervision of Yann Régis-Gianas about Learn-OCaml. It is a great platform designed to learn the OCaml language:

Professors are able to write exercises and grade them automatically by reasoning on the student’s code AST.
Students can experiment OCaml in their web-browser thanks to the js-of-ocaml compiler and a nice UI. It means that grading is performed mostly on the student side, and the teacher can grade files himself only once when he is correcting student’s exercises.

One of my goals this month is to develop a tool to help teachers to determine how exercises were solved by students, that is, to identify “similar” codes, and split the whole exercise database into equivalence classes.

The term “exercises” is quite vague, and for the sake of simplicity, we will consider all exercises as just “defining a function”.

First steps

The first step is to divide answers by their automatic grade: 2 codes with different grades are certainly different. Note that we make the hypothesis that teachers have correctly written their tests (for example, they do not give a random mark for each exercise). If it is not the case, professors will have a bigger problem than partitioning code anyway. Hence, we will focus on how to determine equivalence classes between codes with the same grade.

We have a very strong hypothesis:

All codes are valid OCaml fragments: it is guaranteed that they pass the OCaml type-checker.
Every code has (almost) same semantics because they share the same grade.

Because I am focusing on how an exercise was solved, I have decided to study code’s Abstract Syntax Tree (AST). Indeed, it allows us to abstract from spaces and indentation differences between user codes, and will introduce some semantic into the analysis.

The question becomes now: how do we identify two “similar” AST?

Lambda

Speaking of AST, it is important to say the language it represents. I choose not to work directly on the whole OCaml language, but on an intermediate language used in the compilation pipeline called Lambda. Mainly, Lambda is partially optimized (with inlining of some values, compiled pattern-matching…) and removes syntactic sugar (such as the function key-word), which allows us to abstract even more from the syntax and try to focus on semantic.

Note that, to construct a Lambda expression from an OCaml parse-tree, we need to type-check it. This method is not available for all use-cases, but is fine here since all codes are guaranteed to type-check.

Applied AST fingerprinting

A cool paper

After some research on the Internet, I came across a cool paper: Syntax tree fingerprinting: a foundation for source code similarity detection by Michel Chilowicz, Étienne Duris and Gilles Roussel. 2009, hal-00627811.

The technique presented is pretty simple, while very smart: just hash recursively the AST. It is fast and identical ASTs are guaranteed to share the same hash. But we are not very interested in the direct hash of the structure: for example, two students might have implemented the same code, but with different variable names: we still want to mark the two codes as similar.

The whole point is to find a “good” hash function, that is a function that captures only the interesting part of the AST. The paper rightly suggests that this could be done simply by defining a recursive hash function which doesn’t take into account unneeded data (such as variable names), and only the “shape” of the AST (for example, by hashing a constant string for each node).

In the next paragraph, I will explain what I understood of this paper; we enter after in an original work, about the choice of the clustering analysis of the obtained hashes and an analysis of this choice on a real-world example.

A story of sub-ASTs

Suppose that we have such a “good” hash function. It is still not sufficient, because some codes may share substantial sub-AST, but not be totally equal, and thus the two ASTs will have different hashes. For example, we want to recognize that

if b then x else y

is very close to

if not b then y else x

As suggested in the previously mentioned paper, a possible idea is to keep the hash of “big” sub-ASTs (where the weight of an AST is the number of nodes and leaves it has).

Then, the hash function will now be of type hash : int -> (int * string) list: it takes a weight threshold, and for every node of the AST that is over the threshold, it keeps the weight plus the hash.

The obtained list is after sorted. Because we made the hypothesis that all codes shared the same semantic, we want to identify two codes with a hash list that have the same elements but in a different order (for example, where the order of pattern-matching cases was not the same).

To say it more clearly, we are not interested in little sub-AST alone (because for example, a (partial) function application ( + ) 1 does not give enough information), but in how they are composed (for example, map (( + ) 1) xs). Do not mistake yourself, we do not forget about little sub-AST: because the hash is computed recursively, their hash will be used to compute the hash of their parents. We just don’t take them into account when measuring the distance between two ASTs.

Clustering

We are now able to identify exactly similar ASTs (so, same ASTs up to variable names and syntactic sugar) because they will share the same hash list. But how do we identify “less exactly” similar ASTs, that is, to identify “close” ASTs?

In a mathematical world, this starts by defining something like a distance between two ASTs:

A possible semimetric

let sum_of_fst =
  List.fold_left (fun acc (x,_) -> acc + x) 0

(* Note: None is used to represent infinity *)
let semimetric x y =
  let hx = hash x in
  let hy = hash y in
  if is_intersection_empty hx hy
  then None
  else Some (sum_of_fst (symmetric_difference hx hy))

There is two cases:

If the intersection is empty, then the two set are declared infinitely distant.
Else, we sum the weight of all hashes in the symmetric difference (the symmetric difference of two sets A and B is defined as: (A ∪ B) \ (A ∩ B))

Note that this semimetric is well-defined:

distance x y = 0 <=> x = y, because the symmetric difference of two sets is the empty set if and only if the two sets are the same.
distance x y = distance y x, because the symmetric difference is commutative.

The fact that our hash lists are sorted allows an implementation of symmetric_difference in linear time.

Meaning

What does this “distance” mean? If two ASTs don’t share at least a sub-AST, then they are totally distinct (ie. infinitely distant). Else, we see how many of sub-ASTs they share, compared to all their recorded sub-ASTs. If they share big sub-ASTs, then the sum of the weight of the symmetric difference will be small. Else, if they share only a little sub-AST, then the sum of the weight of the symmetric difference will be bigger.

Cluster analysis

Now that we have a “distance”, we want to group ASTs that are close of each-others. This operation is called a cluster analysis of our hash lists.

I chose to use a hierarchical clustering (which produces a dendrogram) because, mainly, we don’t know how many classes we will obtain, and classical algorithms such as k-means require this information.

The idea is simple: start with every hash list as a class with a single element, then at each step, merge the two closest classes, if it is possible.

The question is: “how do we determine the distance between two classes?”. There are several approaches, but I choose to use the maximum of the distances between 2 elements of the sets, that is, for two classes X and Y: max {distance x y | x ∈ X, y ∈ Y}. This is called a complete-linkage clustering. If the maximum is infinite, then I choose not to merge the two classes.

Mainly, this approach renders the idea that: “if there are two elements that don’t share at least one sub-AST, then the classes cannot be merged”.

The clustering phase returns a list of binary trees representing the classes and how they were made. It allows the end-user to see the distance between two ASTs, and thus get an idea of how good is the relation linking two implementations in the same class.

At the end

Let me sum up the process:

First, we split all codes by their grade.
Then, for each obtained class:
- First we transform all codes into lambda AST.
- Then we hash all of these AST to obtain a list of hash of their sub-ASTs.
- Then we make the classes using the presented hierarchical clustering.

There is only one parameter that needs to be taken into account: the weight threshold for sub-ASTs

Some results

I have tested all of this on the exercises from 3rd year student’s first course of OCaml in my University (a course that I have followed myself this year). The weight threshold is relative to the whole AST weight, and after some experiments, setting it to 30% (that is, only keeping sub-ASTs that have a weight superior to 30% of the whole AST weight) gives good results.

Of course, the results are normally nicely rendered into the Learn-OCaml UI. Here is a simplified version, where is displayed the code of one element in each sub-classes (constituted only of codes with the same hash, thus very close one of each other).

Rev

One interesting exercise was to define rev : 'a list -> 'a list which is reversing a list. My tool split 154 implementations (which all had 10 out of 10) into 9 classes, let us study the main ones:

A main class not so satisfying

The biggest equivalence class has 85 students, split in 7 sub-classes. (The number next to the Node constructor is the actual distance between the two sub-classes).

Node 352.

Leaf with 6 student(s):

let rec rev l = match l with
| [] -> []
| [a] -> a::[]
| a::b -> (rev b) @ [a]

Leaf with 1 student(s)

let rec rev l = match l with
[] | [_] -> l
| t::q -> rev q@[t]

Node 299.

Node 254.

Node 246.

Leaf with 25 student(s)

let rec rev l =
match l with
[]->[]
|a::q -> append (rev q) [a]

Leaf with 1 student(s)

let rec rev l =
match l with
| [] -> l
| t::q -> append (rev q) [t]

Node 196.

Leaf with 49 student(s)

let rec rev l = match l with
|[] -> []
|h::t -> (rev t)@[h]

Leaf with 2 student(s)

let rec rev l = match l with
|[]->l
|t::l' -> (rev l')@[t]

Leaf with 1 student(s)

let rec rev l =
match l with
|[]->[]
|[e]-> l
|e::q -> append (rev q) [e]

I think we can say that the tool effectively captured “close” codes. Even if sometimes there is a not needed pattern-matching, or the use of append (a function defined in a prior exercise) instead of @, the idea is the same.

Note that such a result directly inform the teacher that he needs to explain more the time complexity of the concatenation of lists in his course :)

A second class

The second class is constituted of 38 students:

Node 102.

Leaf with 22 student(s)

let rev l =
let rec rev_acc acc = function
| [] -> acc
| hd::tl -> rev_acc (hd::acc) tl
in
rev_acc [] l

Leaf with 14 student(s

let rec rev ls =
let rec aux xs rs =
match xs with
| [] -> rs
| hd :: tl -> aux tl (hd :: rs)
in aux ls []

Leaf with 2 student(s)

let rev l = match l with
    [] -> []
  | h :: q -> let rec aux new_l l = match l with
        [] -> new_l
      | h :: q -> aux (h :: new_l) q in aux [] l

Again here, the tool captured the idea of an auxiliary function.

The others

The third class is constituted with 12 students who used List.rev or List.fold_left, and the 6 last of other solutions (mainly some variations, using local let bindings, List.hd instead of a pattern-matching, et caetera).

Conclusion

Obtained results are surprisingly good and I think that such a tool can help teachers in their work.

There are many things to do:

The ideas presented here are (almost) purely syntactical and it would be great if we were able to, for example, distinguish the use of a function from the standard library and a function made by the student.
Another possibility is to develop an AI, trained by teachers, to refine the partitioning made. For example, a teacher could say: these two AST looks syntactically equivalent, but their semantic is not the same, and they should not be in the same class.
The implementation I made focus on one toplevel definition, thus it does not take into account auxiliary definition made outside of the function.
Other clustering approach can maybe give better results
A better study of the weight threshold could also help to optimize the results.

Thanks

I want to thank Yann Régis-Gianas for his proofreading and his wise advise on this article.

Paramorphisms fusion for algebraic graphs

2019-02-03T00:00:00+01:00

Introduction

Algebraic graphs are a wonderful structure to work with, and especially to learn optimization techniques!

In this article, the goal is to optimize at compile-time the composition of some paramorphisms and homomorphisms of graphs (don’t worry with these barbaric terms, all of this is discussed below).

If you never approached algebraic graphs, I suggest that you read the original paper by A. Mokhov, my tutorial or just the documentation on Hackage of Alga, the Haskell library that implements the idea.

The problem

Shortcut deforestation

This process is introduced in “A Short Cut to deforestation” (by A. Gill, J. Launchbury and S. Peyton Jones). Basically, it is the idea that one can remove intermediate structures when working with “good functions” (functions that are preserving the structure of the object on which they are working on).

For example, the expression

foldr (+) 0 . map (+1)

builds an intermediate (and unneeded) list: map will build a new list that will be destructed right away by foldr. It can be optimized (and GHC does it) to

foldr (\x y -> x + 1 + y) 0

Note that this is possible only because map preserves the list structure.

On algebraic graphs, we have an equivalent to foldr named foldg:

data Graph a =
    Empty
  | Vertex a
  | Overlay (Graph a) (Graph a)
  | Connect (Graph a) (Graph a)

foldg :: b -> (a -> b) -> (b -> b -> b) -> (b -> b -> b) -> Graph a -> b
foldg e v o c = go
  where
    go Empty         = e
    go (Vertex  x  ) = v x
    go (Overlay x y) = o (go x) (go y)
    go (Connect x y) = c (go x) (go y)

Consider the following expression

foldg e v o c (f x)

where f :: Graph a -> Graph b is an operation that preserves the structure of the graph (mathematicians like to call this an homomorphism) and x is a graph.

We are pretty much in the same situation as for lists: indeed, the above expression currently goes through the structure two times (one for f, the other for foldg). The goal is to rewrite it to an equivalent expression that goes through the graph only once, using the fact that f is an homomorphism.

Implementation for algebraic graphs

I already wrote about how I implemented this technique for foldg using GHC’s rewrite rules. I suggest that you read it to understand better the following, even if I will try to quickly summarize it just below.

Quick reminder

The basic idea is to view a graph homomorphism as a function transforming arguments passed to foldg (which are generally graph constructors).

First of all, we need to remember a cool result (proved in this blog): graphs homomorphisms are exactly functions that are made with bind, which is written for graphs as:

(>>=) :: Graph a -> (a -> Graph b) -> Graph b
x >>= f = foldg Empty f Overlay Connect x

So we only have to optimize something like:

foldg e v o c (x >>= f)

The question is: how to represent “functionally” the bind operation? Remember, if we can represent bind as a function operating on foldg arguments, we can easily fuse it with an actual foldg-function (imitating what GHC does for lists).

First, there is a type that encapsulate this notion:

type Foldg a = forall b. b -> (a -> b) -> (b -> b -> b) -> (b -> b -> b) -> b

Then we can use these rules:

buildR :: Foldg a -> Graph a
buildR g = g Empty Vertex Overlay Connect
{-# INLINE [1] buildR #-}

-- These rules transform functions into their buildR equivalents.
{-# RULES
"buildR/bindR" forall f g.
bindR g f = buildR (\e v o c -> foldg e (foldg e v o c . f) o c g)

"foldg/buildR" forall e v o c (g :: Foldg a).
foldg e v o c (buildR g) = g e v o c
 #-}

It effectively transforms any occurrence of bind into a “function-like” foldg, that transforms its arguments.

Then if we encounter an expression like foldg e v o c . buildR, we can optimize the composition using "foldg/buildR"!

If there was nothing to optimize, things are rewritten back to their normal form with the inlining of buildR.

hasEdge

We can now optimize the composition of several functions of the Alga library.

Let me introduce a killjoy: the hasEdge function, that tests if an edge is present in the graph. It is actually defined as:

data Hit = Miss | Tail | Edge deriving (Eq,Ord)

hasEdge :: Eq a => a -> a -> Graph a -> Bool
hasEdge s t g = hit g == Edge
  where
    hit Empty         = Miss
    hit (Vertex x   ) = if x == s then Tail else Miss
    hit (Overlay x y) = case hit x of
        Miss -> hit y
        Tail -> max Tail (hit y)
        Edge -> Edge
    hit (Connect x y) = case hit x of
        Miss -> hit y
        Tail -> if hasVertex t y then Edge else Tail
        Edge -> Edge

The algorithm is pretty simple:

There is of course no edge in an Empty graph, so you Miss it.
If you hit a Vertex corresponding to the Tail of the edge, you say it, else you Miss.
For an Overlay node:
- If you missed it in the left part, the edge can only be in the right one.
- If you hit a Tail in the left part, maybe there is the whole edge in the right part, but at least you hit the Tail.
- If you hit an Edge, you say it!
For a Connect node,
- If you missed it in the left part, the edge can only be in the right one.
- If you hit a Tail in the left part:
  - If the right part contains the other part of the edge, you found the whole Edge.
  - Otherwise, you only have its Tail.
- If you hit an Edge, you say it!

Observations

hasEdge seems a bit like a foldg-function but not totally. The problem comes with the call to hasVertex on an unconsumed part of the graph in the middle of the process.

Our actual rules will not optimize hasEdge (x >>= f), since our hasEdge is not expressed with foldg.

hasEdge seems anyway not so extraordinary and it is easy to see that it can be indeed optimized when used with fmap to traverse the graph only once. For example hasEdge s t (fmap f g) is equivalent to:

hasEdgeF :: Eq b => (a -> b) -> Graph a -> b -> b -> Bool
hasEdgeF f g s t = Edge == go g
  where
    go Empty = Miss
    go (Vertex x) = if (f x) == s then Tail else Miss
    go (Overlay x y) = case go x of
        Miss -> y
        Tail -> max Tail y
        Edge -> Edge
    go (Connect x y) = case go x of
        Miss -> y
        Tail -> if foldg False (\x -> f x == t) (||) (||) y then Edge else Tail
        Edge -> Edge

Note that in this expression, the fmap operation was incorporated in go (Vertex x) and in the hasVertex call (which was also inlined).

Can the theory help us to optimize it automatically? And even better, for any function like hasEdge, that are looking almost like foldg but not?

More theory

Of bananas and barbed wire

The great article “Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire” of E. Meijer, M. Fokkinga and R. Paterson, describes some recursion operators; here, we will focus on two major: catamorphisms (written between bananas bracket in the article) and paramorphisms (written between barbed wire bracket in the article). If you haven’t read it, it is really worth it!

Catamorphisms

Catamorphism (which comes from Greek “κατα” meaning “downwards”, “down from”) is a term to describe functions that:

In functional programming, generalize the fold function, that is, on a list, a function of type cata :: (a -> b -> b) -> b -> [a] -> b. cata take a combination function, a base case, and fold the structure accordingly.
From a mathematical point of view, it (quoting the great Wikipedia) “denotes the unique homomorphism from an initial algebra into some other algebra”. Here, the important word is, I think, “homomorphism”: being a homomorphism, a catamoprhism will preserve the underlying algebraic structure.

You already know cata for lists; as a clever reader has already noticed from its type, cata for lists is nothing more than foldr in disguise:

cata :: (a -> b -> b) -> b -> [a] -> b
cata combine e []     = e
cata combine e (y:ys) = combine y (cata k z ys)

Did you guess what is the function expressing a catamorphism on graph? It is foldg!

Paramorphisms

Quoting again Wikipedia: “In formal methods of computer science, a paramorphism (from Greek παρά, meaning “close together”) is an extension of the concept of catamorphism […] to deal with a form which ‘eats its argument and keeps it too’”.

The idea is to extend the combination function: you can allow it to reason on the structure itself. When you are doing recursion, you sometimes want to inspect (or directly use) what is left of the structure (note that you cannot simply do that in a catamorphism (since this argument is not available to the combination function)).

For example, again on lists, you will have something like: para :: (a -> ([a], b) -> b) -> b -> [a] -> b. The combination function can use the not already consumed list.

para :: (a -> ([a], b) -> b) -> b -> [a] -> b
para combine e []     = e
para combine e (y:ys) = combine e (y,ys) (para k z ys)

But why can I need such a function ? foldr does already anything I want!

Indeed foldr is equivalent (in term of expressivity) to para. This is nicely explained in this stackoverflow answer. Basically, you can define foldr with para and you can define para from foldr. But if you do it with para' (again from the mentioned stackoverflow answer):

para' c n = snd . foldr (\ x (xs, t) -> (x : xs, c x xs t)) ([], n)

para' will be slower than the one we defined: it builds an exact copy of the list as it traverses it.

So the answer to the above question is efficiency: para allows you to make more efficient functions, and when you work with graphs (and when you simply work), you just want the more efficient implementation!

Paramorphisms for graphs

One can make paramorphisms for graphs by changing a bit foldg (and making more complex its signature):

paragraph :: b -> (a -> b) -> (b -> b -> Graph a -> Graph a -> b) -> (b -> b -> Graph a -> Graph a -> b) -> Graph a -> b
paragraph e v o c = go
  where
    go Empty         = e
    go (Vertex  x  ) = v x
    go (Overlay x y) = o (go x) (go y) x y
    go (Connect x y) = c (go x) (go y) x y

(Thanks Andrey for the great name :) ).

Back to `hasEdge`

You guessed it, hasEdge is a paramorphism!

Indeed we can write hasEdge using paragraph:

hasEdge :: Eq a => a -> a -> Graph a -> Bool
hasEdge s t = (==) Edge . paragraph Miss v o c
  where
    v x = if x == s then Tail else Miss
    o x y _ _ = case x of
      Miss -> y
      Tail -> max Tail y
      Edge -> Edge
    c x y _ yy = case x of
      Miss -> y
      Tail -> if hasVertex t yy then Edge else Tail
      Edge -> Edge

The goal

Our current goal is to optimize

paragraph e v o c (bind f x)

to something that traverses the original graph x only once. If we achieve this, it will solve our problem with hasEdge and for every function expressed with paragraph.

Is this even possible?

A legitimate question is: can we indeed do such a thing? Is not paragraph too general to allow fusion?

This is not so simple and I did not found an optimization that works for every paramorphism (and I don’t think it is possible), but only for a subclass of them.

I proved in Coq the following theorem (that is valid for every paramorphism):

Definition apply2L {A B C D}
  (f : A -> A -> D -> D -> C) (g:B -> D) :=
  fun a b x y => f a b (g x) (g y).

Theorem para_compo
  {A B C}
  {e : C} {v: B -> C} {o:C -> C -> Graph B -> Graph B -> C} {c:C -> C -> Graph B -> Graph B -> C}
  {f : A -> Graph B}:
paragraph e v o c ∘ bind f =
paragraph e (paragraph e v o c ∘ f) (apply2L o (bind f)) (apply2L c (bind f)).

(∘ stands for functional composition in Coq).

With the right part of the equality, you will traverse the graph once using paragraph until you find a vertex, and then you will apply your “binded” function and traverse the result with paragraph again.

The only “little” problem is with apply2L. Indeed, you need to change the last arguments of your paramorphism to reflect the bind in the unconsumed part of the graph.

You can view this by a duplication of the bind operation: it gets incorporated in the paragraph expression but also for the two lasts arguments of the combination function. This leads to a simple problem: you don’t want to rewrite functions that use both the unconsumed graph and the recursive call. If you allow this, the rewriting will lead to a double computation of the “binded” function (one for the recursive call, and one for the delayed bind on the unconsumed arguments).

We need to ensure that we rewrite only functions that make the promise to use either the recursive call or the unconsumed graph. Note that hasEdge fulfill this promise, so all is well!

Implementation

Promises need to be kept

I don’t like “functions that make promises”, so we need to encode the promise “I either use the recursive call or the unconsumed graph” in types.

The not working but elegant version

Ideally, this kind of function can be encoded with:

type Combination a b =
  Either
    (b -> Either (b -> b) (Graph a -> b))
    (Graph a -> Either (b -> b) (Graph a -> b))

First of all, this indeed encode the promise, almost literally: your function either use the left recursive call or the left unconsumed part and then produce a function that either use the right recursive call or the unconsumed right graph.

But with this definition comes a problem: the second argument of the combination function is “hidden” in another function. For example, with such a definition, the only way to define apply2L is something like:

apply2L :: (Graph c -> Graph a) -> Combination a b -> Combination c b
apply2L c (Left f) = Left $ \a ->
  case f a of
   Left f'  -> Left $ \b -> f' b
   Right f' -> Right $ \y -> f' (c y)
apply2L c (Right f) = Right $ \x ->
  case f (c x) of
    Left f'  -> Left $ \b -> f' b
    Right f' -> Right $ \y -> f' (c y)

Your only choice (and this is logical) is to apply first the function (either on the recursive call of the unconsumed graph) and discuss its result. Sadly, this way will lead to a problem: one cannot fuse any operation (like bind) on the unconsumed right part of the graph. GHC is “blinded” by the function encoding and cannot inline what the returned function (f') really is (since it depends at run-time on the result of f).

A solution to this problem is to write explicitly the combination function, giving the user only the control on some parameter.

A working solution

Here is a solution, not very classy, but working (if you find something better, don’t hesitate to tell me) :

data Combination a b =
    B (b -> b -> b)
  | L (b -> Bool, b -> b -> b, Graph a -> b -> b)
  | R (b -> Bool, b -> b -> b, b -> Graph a -> b)

You have three possibilities:

B: it is a classic combination function (like the one we can find in foldg).
L: you inspect the left recursive call using a predicate b -> Bool, decide what to do, either call the recursion on the right part or inspect the right unconsumed part.
R: the same, but you inspect first the right recursive call.

Here is switchCombi that effectively transform this data into a combination function for paragraph

switchCombi :: Combination a b -> b -> b -> Graph a -> Graph a -> b
switchCombi (B f) = \a b _ _ -> f a b
switchCombi (L (pred,c1,c2)) = \a b x _ ->
  if pred b
  then c1 a b
  else c2 x b
switchCombi (R (pred,c1,c2)) = \a b _ x ->
  if pred a
  then c1 a b
  else c2 a x

Note that providing a constructor for the case Graph a -> Graph a -> b is not very interesting, since it does not use any recursive calls.

Thus we can define paragraphR:

paragraphR :: b -> (a -> b) -> Combination a b -> Combination a b -> Graph a -> b
paragraphR e v o c = paragraph e v (switchCombi o) (switchCombi c)
{-# INLINE [0] paragraphR #-}

Then it is easy to define hasEdge using paragraphR :

hasEdge :: Eq a => a -> a -> Graph a -> Bool
hasEdge s t = (==) Edge . paragraphR Miss v (B o) (R (cp,c1,c2))
  where
    v x = if x == s then Tail else Miss
    o x y = case x of
      Miss -> y
      Tail -> max Tail y
      Edge -> Edge
    cp x = x /= Tail
    c1 x y =
      case x of
        Miss -> y
        _ -> x
    c2 _ y = if hasVertex t y then Edge else Tail

Foldg adaptation

To adapt this for paragraphR we need to change the previous encoding of Foldg with

Paragraph a =
  forall b. b -> b -> (a -> b) -> Combination a b -> Combination a b -> b

That is a structure that represents exactly the type of paragraphR (like Foldg represented foldg).

Given a function into its Paragraph form, it is easy to get the graph it represents:

buildR :: Paragraph a -> Graph a
buildR g = g Empty Vertex (B Overlay) (B Connect)
{-# INLINE [1] buildR #-}

Bind arguments

Some problems come with the choices we made, lets see directly the rules and some auxilliary functions (don’t run away!):

apply2LR :: (Graph c -> Graph a) -> Combination a b -> Combination c b
apply2LR _ (B a) = B a
apply2LR f (L (a,b,c)) = L (a,b,\x y -> c (f x) y)
apply2LR f (R (a,b,c)) = R (a,b,\x y -> c x (f y))

-- Bind already in its "buildR" form (to avoid a rewriting cycle with "buildR/bindR")
buildBindR :: (a -> Graph b) -> Graph a -> Graph b
buildBindR f g =
  buildR
    (\e v o c ->
	  paragraphR
	    e
	    ((paragraphR e v o c) . f)
	    (apply2LR bind2R o)
	    (apply2LR bind2R c)
	    g
    )
  where
    -- bind without rewrite rules (to avoid a rewriting cycle with "buildR/bindR")
    bind2R = foldg Empty f Overlay Connect
{-# INLINE buildBindR #-}

{-# RULES
"buildR/bindR" forall (f::a -> Graph b) g.
  bindR g f =
    buildR
      (\e v o c ->
	    paragraphR
	      e
	      ((paragraphR e v o c) . f)
	      (apply2LR buildBindR o)
	      (apply2LR buildBindR c)
	      g
      )

"foldg/buildR" forall e v o c (g :: Paragraph a).
  foldg e v o c (buildR g) = g e v (B o) (B c)

"paragraphR/buildR" forall e v o c (g :: Paragraph a).
  paragraphR e v o c (buildR g) = g e v o c
 #-}

Far more scary than the previous ones, isn’t it?

Let me explain what are doing the rules:

"buildR/bindR" is converting a bind into its buildR equivalent (it is, up to some names, exactly the previous theorem).
"foldg/buildR" assure the compatibility with the previous work: you can always fuse a foldg on the left of a bind!
"paragraphR/buildR" is the main rule and fuse a paragraphR on the left of a bind.

The complication comes from the fact that we fuse with bind. Because we allow the combination function of paragraph to reason on the graph itself, we will have to remember the bind operation that needs to be made on the “not already consumed argument” of our paramorphism (more explicitly, we are delaying the bind operation). This is done by apply2LR which apply bind to the two lasts arguments.

Even with this, the above rules seem a lot more complicated than necessary.

Warning: cycle

One (who does not read commentaries) might ask

Why do you use buildBindR and not directly bind?

Because there is a little problem. The "buildR/bindR" rules is precisely rewriting bind occurrences, so using it directly in the right part of the equality will lead to a rewriting cycle.

BUT

We need this bind and we need it in a “re-writable form”. If you look at hasEdge, you see that it uses hasVertex on the unconsumed part of the graph, and this function can be fused with bind. So, ideally, standard shortcut deforestation will also happen here, directly on the argument of combination function for Connect.

The only possibility is to use buildBindR, which is only bind is its buildR form. We use the direct code of bind for the arguments, because we do not want to come into a rewriting cycle.

Conclusion

Is all of this working?

Yes! Using inspection-testing (a great tool that compare the code produced by GHC at compile-time), the following code compiles:

hasEdgeF :: Eq b => (a -> b) -> Graph a -> b -> b -> Bool
hasEdgeF f g s t = hasEdge s t (fmap f g)

hasEdgeFR :: Eq b => (a -> b) -> Graph a -> b -> b -> Bool
hasEdgeFR f g s t = Edge == go g
  where
    go Empty         = Miss
    go (Vertex  x  ) = if (f x) == s then Tail else Miss
    go (Overlay x y) =
      case go x of
        Miss -> go y
        Tail -> max Tail (go y)
        Edge -> Edge
    go (Connect x y) =
      let gx = go x in
        if gx /= Tail
        then
          case gx of
            Miss -> go y
            _ -> gx
        else if foldg False (\x -> f x == t) (||) (||) y then Edge else Tail

-- If this compiles, it means that hasEdge and hasEdgeFR have (almost) the same
inspect $ 'hasEdgeF === 'hasEdgeFR

What happened?

The fmap was entirely fused with hasEdge. By entirely, I mean not only the test on vertices, but also the composition of hasVertex and fmap happened!

More precisely, the fmap operation is made only on the needed part of the graph. That is, if your graph is like Overlay a b where the desired edge is on the a part, the b part of the graph will not be evaluated (and particularly not the fmap operation on this part).

What remains to be done

Add list optimizations: a classic way to create an algebraic graph is to use a list (an adjacency list for example). For example edges :: [(a,a)] -> Graph a that make an algebraic graph from a list of edges. We already have some rewrite rules to optimize expressions like foldg e v o c (edges xs) (because we know how the graph is made, we can avoid building an intermediate graph). Is this possible with paragraph?
Alga: all I described here is already implemented on my fork of Alga, but there is certainly some cleanup and verifications to make it available in the official Alga repository!
Proofs: I only proved that we can optimize things. It will be nice to actually prove that the rewriting process is correct (that is: we rewrite things only to equivalent ones).

Thanks

Thanks to Andrey Mokhov for his review of this article.

Some results about algebraic graphs homomorphisms

2018-12-09T00:00:00+01:00

Prelude

In my previous post “Rewrite rules and a specific fold: use optimization techniques from GHC.Base”, I described a technique to optimize composition of a custom fold and some functions on algebraic graphs, implemented in Haskell in the alga library.

The goal of this post is to try to specify exactly what are the classes of functions we can optimize. I will try to do this in a “mathematical” way (don’t run, this is actually fun), with all my results proved in Coq, a very cool proof assistant (it is a first time for me, please be indulgent with my code :) ).

All the work can be found at https://github.com/nobrakal/coq-alga/.

The version of this repositiory when I wrote this article can be found at https://github.com/nobrakal/coq-alga/tree/4586bda0847b059b263cc4a1c8685004c9461d6c.

Alga theory

I had to implement some of the already known algebraic graphs theory in Coq (already know because proved in agda on a dedicated repository).

All of this work is in the src/Graph.v file:

The data structure, an inductive one called Graph.
Some standards functions (foldg, isEmpty…).
Useful lemmas on them.
Axioms on the graph equality (in a Coq class called EqG).
Indications for Coq to allow rewriting (using the rewrite tactic) when working with equal graphs.
Some proofs about the graph equality (especially, that Empty is the identity for Overlay).

A note on equality

Here the = symbol is ambiguous.

So we will distinguish:

a = b for the structural equality (a.k.a. Leibniz equality) between a and b.
R a b for graph equality relation between a and b (note that the axioms of this relation are satisfied by others relations than graph equality, so things are not so simple).

Homomorphism

The work described here can be found in the src/Homomorphism.v file.

Graph homomorphism

An homomorphism in algebra theory is (quoting Wikipedia) “a structure-preserving map between two algebraic structures of the same type”.

For us, a correct homomorphism f :: Graph a -> Graph b should satisfy:

f Empty = Empty
f (Overlay a b) = Overlay (f a) (f b)
f (Connect a b) = Connect (f a) (f b)

This is encoded in a Coq class called Homomorphism:

Class Homomorphism {A B:Type} (f : Graph A -> Graph B) : Prop := {
  Hom_Empty :> f Empty = Empty;
  Hom_Overlay :> forall a b, f (Overlay a b) = Overlay (f a) (f b) ;
  Hom_Connect :> forall a b, f (Connect a b) = Connect (f a) (f b)
 }.

Equivalence with bind-functions

Let us recall briefly the concerned Haskell functions:

data Graph a = Empty
             | Vertex a
             | Overlay (Graph a) (Graph a)
             | Connect (Graph a) (Graph a)

foldg :: b -> (a -> b) -> (b -> b -> b) -> (b -> b -> b) -> Graph a -> b
foldg e v o c = go
  where
    go Empty         = e
    go (Vertex  x  ) = v x
    go (Overlay x y) = o (go x) (go y)
    go (Connect x y) = c (go x) (go y)

(>>=) :: Graph a -> (a -> Graph b) -> Graph b
g >>= f = foldg Empty f Overlay Connect g

A cool result is that such homomorphisms are exactly bind-functions, that is to say any graph homomorphism can be expressed with the (>>=) operator.

Proof

So any function expressed with (>>=) is clearly a graph homomorphism (this is proved in the bind_is_hom lemma).

A fun fact is that the reciprocal is also true, like shown in the hom_is_bind lemma:

Lemma hom_is_bind A B (hom: Graph A -> Graph B):
  Homomorphism hom -> hom = bind (hom ∘ Vertex).

That is, an homomorphism is equal to bind with its definition for a vertex. A graph homomorphism is indeed entirely defined by what it is doing on a leaf of the graph expression.

Examples

fmap f is a graph homomorphism, using Vertex . f as bind argument.
removeVertex v is a graph homomorphism, using \x -> if v == x then Vertex x else Empty as bind argument.

Theorem 1: Composition of a graph homomorphism and a foldg function

Any graph homomorphism can be fused with a foldg-function, like show in the foldg_bind theorem:

Theorem foldg_bind A B C (e : C) (v: B -> C)
  (o:C -> C -> C) (c:C -> C -> C) (f_v : A -> Graph B):
    foldg e v o c ∘ bind f_v = foldg e (foldg e v o c ∘ f_v) o c.

Basically, because a graph homomorphism preserve the structure, it can easily be fused with something that destruct this structure: this is shortcut deforestation.

Theorem 2: Composition of graph homomorphisms

A cool corollary of the previous theorem is that the composition of two graph homomorphisms is again a graph homomorphism.

Proof

This is proved in the bind_compo theorem:

Theorem bind_compo A B C (hom1 : Graph A -> Graph B) (hom2 : Graph B -> Graph C):
 (Homomorphism hom1) /\ (Homomorphism hom2) ->
 hom2 ∘ hom1 = bind (hom2 ∘ hom1 ∘ Vertex).

Problems

So graph homomorphisms are cool, but not always what we want: in the examples of graph homomorphisms, removeVertex is leaving empty leaves in the structure. If it was removing them (like the actual removeVertex in the alga library), it will change the graph structure, and thus will not be a graph homomorphism as we defined it. Such functions are very useful, so we must lower our restrictions and only require preservation of the graph equality to include them.

Reduced graph homomorphism

Theorems of this section are in the src/ReducedHomo.v file.

Definition

A function f :: Graph a -> Graph b is called a reduced graph homomorphism if:

R (f Empty) Empty
R (f (Overlay a b)) (Overlay (f a) (f b))
R (f (Connect a b)) (Connect (f a) (f b))

So we require only that the function has the same properties than a graph homomorphism, but with graph equality instead of the structural one.

This idea is encapsulated in the Reduced_hom Coq class:

Class Reduced_hom {A B : Type} (R: relation (Graph B)) (f : Graph A -> Graph B) : Prop :=
{
  RHom_EqG :> EqG B R;
  RHom_Empty :>  R (f Empty) Empty ;
  RHom_Overlay :> forall a b, R (f (Overlay a b)) (Overlay (f a) (f b)) ;
  RHom_Connect :> forall a b, R (f (Connect a b)) (Connect (f a) (f b))
}.

Theorem 3: Reduced graph homomorphism are indeed reduced graph homomorphism

Graph homomorphism are reduced graph homomorphism.

The reverse is false.

Proof

“graph homomorphism are reduced graph homomorphism.”

This is proved in the hom_is_reduced_hom theorem.

The idea is only to use the fact that, for all a b :: Graph A, if a = b then R a b.

“The reverse is false”. Using

wforget :: Graph a -> Graph b
wforget _ = Empty

wforget is clearly a reduced homomorphism (this is proved in the const_empty_is_reduced_hom theorem).

But it is also clearly not a graph homorphism, this is proved by the absurd in the const_empty_is_not_hom theorem.

The case of empty leaves: a class of reduced graph homomorphism

Theorems of this section are in the src/SmartHomo.v file.

Prelude

-- Remove empty leaves around a graph operator (usually Overlay or Connect)
kSimpl :: (Graph a -> Graph a -> Graph a) -> Graph a -> Graph a -> Graph a
kSimpl c x y =
  if isEmpty x
  then if isEmpty y
    then Empty
    else y
  else if isEmpty y
    then x
    else c x y

-- Remove empty leaves of a graph
dropEmpty :: Graph a -> Graph a
dropEmpty = foldg Empty Vertex (kSimpl Overlay) (kSimpl Connect)

Definition

A function f :: Graph a -> Graph b is called a smart graph homomorphism if it can be described using f_w :: a -> Graph b such that

f g = dropEmpty (g >>= f_w)

Obviously, such a function can go through the graph only once, using

f = foldg Empty f_w (kSimpl Overlay) (kSimpl Connect)

This is proved in the smart_hom_single theorem.

Example

induce is a smart graph homomorphism (maybe the definition was made for it…):

induce :: (p -> Bool) -> Graph a -> Graph a
induce p =
  foldg
    Empty
    (\x -> if p x then Vertex x else Empty)
    (kSimpl Overlay)
    (kSimpl Connect)

This is proved in the induce_smart_hom theorem.

Theorem 4: smart graph homomorphisms are reduced homomorphisms.

This seems clear because empty leaves does not play any role in graph equality, but it takes me anyway 44 lines to prove it.

It is proved in the smart_hom_is_reduced_hom theorem:

Theorem smart_hom_is_reduced_hom A B (R: relation (Graph B))
  (f : Graph A -> Graph B) :
    EqG B R -> Smart_hom f -> Reduced_hom R f.

Theorem 5: Composition of smart graph homomorphisms

The composition of two smart graph homomorphisms is again a smart graph homomorphism.

Proof

This was hard to prove because not clear at first glance (well mostly, it was long, using 280 lines ^^). The idea is to see that the composition of two smart graph homomorphisms is something not so strange:

Theorem smart_hom_compo A B C (s1 : Graph A -> Graph B)
  (s2 : Graph B -> Graph C): (Smart_hom s1) /\ (Smart_hom s2) ->
  s2 ∘ s1 = foldg Empty (s2 ∘ s1 ∘ Vertex) (kSimpl Overlay) (kSimpl Connect).

(The proof is here).

Then see that this form allow us to prove that the composition is again a smart graph morphism:

Theorem smart_hom_compo_is_smart A B C (s1 : Graph A -> Graph B)
  (s2 : Graph B -> Graph C):
    (Smart_hom s1) /\ (Smart_hom s2) -> Smart_hom (s2 ∘ s1).

(The proof is here).

Conclusion

So all this work to prove that we can optimize anything like:

foldg e v o c . morph1 . morph2 . [...] . morphn

to a function that will only go through the graph once.

if morph1, morph2, …, morphn are graph homomorphisms (theorem 1 and theorem 2).

We can also optimize any combination of smart graph homomorphisms (theorem 5).

What is left

There is certainly some things to do playing with these results:

Find a cool (and efficient) implementation for the alga library.
The composition of smart graph homomorphisms is preserving graph structure, maybe there is an efficient one that is only preserving graph equality ?
Determine precisely the class of reduced homomorphisms. That is to say, is there an equivalence with a well known class of functions like for graph homomorphisms ?
What is the class of reduced homomorphisms that we can compose ?
Can this be generalised to arbitrary algebraic data types that have some notion of an Empty constructor?
And certainly many other things :)

Thanks

Thanks to Andrey Mokhov for his review of this article.

Rewrite rules and a specific fold: use optimization techniques from GHC.Base

2018-09-30T00:00:00+02:00

Introduction

The GHC.Base module defines, in particular, base functions on lists and several rewrite rules to optimize them (if you are not familiar with rewrite rules, I suggest the reading of my previous article “GHC, one compiler to RULE them all”, or directly the related GHC documentation). Especially, these rules allow

foldr f n (map g xs)

to be rewritten by the compiler into something similar to

foldr (f . g) n xs

These rules improve the expression by allowing a single reading of the list, whereas the first implementation was reading it twice.

Alga, a functional implementation of graphs, defines a foldable structure with a fold (named foldg) specialized for the graph data.

Can we use the same tricks than GHC.Base to optimize compositions of foldg with fmap? Spoiler: Yes, and we can do it without any pain!

GHC.Base optimization on lists

The build function

build is a function you can find in GHC.Base module:

build :: forall a. (forall b. (a -> b -> b) -> b -> b) -> [a]
build g = g (:) []
{-# INLINE [1] build #-}

It is mainly used for optimization when working on list.

For the next part of this article, I suggest you to read this great article by David Luposchainsky (alias quchen): https://github.com/quchen/articles/blob/master/build.md, that mainly explains how the build function works, but not fully how it is used.

I will assume now that you are familiar with build and with lambda-lists.

FB functions

FB functions can be seen when searching for build in GHC.Base. For example, with map there is a mapFB:

mapFB :: (elt -> lst -> lst) -> (a -> elt) -> a -> lst -> lst
mapFB c f = \x ys -> c (f x) ys
{-# INLINE [0] mapFB #-}

Given a c(ons) function and something to convert a generic type a to an element of the list of type elt (namely f), you can append an element of a generic type a to a lambda-list of type elt.

For example

let myLambdaList = \cons nil -> 1 `cons` (2 `cons` (3 `cons` nil))
 in (\cons nil -> mapFB cons read "0" (myLambdaList cons nil) ) (:) []

will produce

[0,1,2,3]

But what is the purpose of these functions since a clever reader will have seen that they are not exported? Optimization! Take a look at the type of mapFB if we apply to it the two first arguments: now you have a convenient function to use in a foldr.

And why bother to give it a name? Because it will allow us to remember what we were rewriting, and rewrite back function when we were not able to make any optimization.

The rules

Let us jump quick in the interesting part. Here is (a part of) the rewrite rules used in GHC.Base to optimize the code of functions working on lists:

{-# RULES
"map"     [~1] forall f xs. map f xs = build (\c n -> foldr (mapFB c f) n xs)

"fold/build"   forall k z (g::forall b. (a->b->b) -> b -> b) .
               foldr k z (build g) = g k z

"mapFB"        forall c f g. mapFB (mapFB c f) g = mapFB c (f.g)

"mapList" [1]  forall f. foldr (mapFB (:) f) [] = map f
  #-}

So what is happening ?

Only in phase 2 (the first one), we rewrite every occurrence of map by its strange build equivalent (which use mapFB), hoping for fusion to happen.
This fusion can happen with two rules, always activated:
- The "fold/build" one, described in the previously mentioned David’s article, merging a foldr followed by a build (thus allowing the simplification of a foldr f n . map g expression).
- If there was two mapFB (and thus something like map f . map g at the beginning of the process), we merge them into a simpler and faster map (f . g)
- In the meantime, build is inlined, allowing the "mapList" rule to fire.
If there is any “dumb” foldr and mapFB left, we rewrite them back to a standard form using the "mapList" rule (only active from the phase 1 to the end).

For more details, see the related comment in the GHC.Base source.

Is it doing it right?

Let us take an example.

Imagine I want to make the sum the elements of a list of integers and add the whole length of this list. One approach (rather bad) is to add 1 to each element of the list and then make the sum its elements. Here is roughly the different phases of the rewriting work made by the compiler if you try to compile such a function:

foldr (+) 0 (map (+1) xs)
=(map)=          foldr (+) 0 (build (\c n -> foldr (mapFB c (+1)) n xs))
=(fold/build)=   (\c n -> foldr (mapFB c (+1)) n xs)) (+) 0
==               foldr (mapFB (+) (+1)) 0 xs
=(inline mapFB)= foldr (\x -> (+) (x+1)) 0 xs

Et voila. At the beginning we were reading two times the list and at the end, only one.

Back to graphs

The data

The main data we will work with is Graph from Algebra.Graph module:

data Graph a = Empty
             | Vertex a
             | Overlay (Graph a) (Graph a)
             | Connect (Graph a) (Graph a)

We can define a fold specialized for Graph:

-- Generalised 'Graph' folding: recursively collapse a 'Graph' by applying
-- the provided functions to the leaves and internal nodes of the expression.
-- The order of arguments is: empty, vertex, overlay and connect.
foldg :: b -> (a -> b) -> (b -> b -> b) -> (b -> b -> b) -> Graph a -> b
foldg e v o c = go
  where
    go Empty         = e
    go (Vertex  x  ) = v x
    go (Overlay x y) = o (go x) (go y)
    go (Connect x y) = c (go x) (go y)
{-# INLINE [0] foldg #-}

This useful function allows us to define two key-instances:

instance Functor (Graph a) where
	fmap f = foldg Empty (Vertex . f) Overlay Connect

instance Foldable (Graph a) where
	foldMap f = foldg mempty f (<>) (<>)

Déjà Vu?

Now if we define for convenience:

mapG :: (a -> b) -> Graph a -> Graph b
mapG = fmap
{-# INLINE [0] mapG #-}

We cannot use a rule targetting directly fmap because it does not have any inline pragma attached to its definition and thus can be inlined at any moment. mapG’s INLINE pragma guarantee that we can target mapG in a rewrite rule during the phase 2 and 1.

We are almost in the same position as when working with lists! We have a foldr and a map. Can we achieve the same as for lists? Can we optimize

foldg e v o c . mapG f

into

foldg e (v . f) o c

using the same method than in GHC.Base for lists?

Introducing buildG

The work is simple, we only need to adapt the type from GHC.Base to graphs:

type GraphF a b = b -> (a -> b) -> (b -> b -> b) -> (b -> b -> b) -> b

buildG :: forall a. (forall b. GraphF a b) -> Graph a
buildG g = g Empty Vertex Overlay Connect
{-# INLINE [1] buildG #-}

A “lambda-graph” of type a is a function of type GraphF a b. So like lambda-lists, applying the four constructors allow us to convert it back to a standard Graph.

Add FB function

Our mapGFB will be slightly different:

mapGFB :: (b -> c) -> (a -> b) -> a -> c
mapGFB = (.)
{-# INLINE [0] mapGFB #-}

It is only a composition of function, but it will allow us to remember that it was made for a rewriting of mapG.

The rules

Now we have all the bricks to build our graph rules:

{-# RULES
-- Transform a mapG into its build equivalent
"mapG"   [~1]   forall f g. mapG f g = buildG (\e v o c -> foldg e (mapGFB v f) o c g)

-- Merge a foldg followed by a buildG
"foldg/buildG"  forall e v o c (g::forall b. GraphF a b).
                       foldg e v o c (buildG g) = g e v o c

-- Merge two mapFB
"mapFB/mapFB"   forall c f g. mapGFB (mapGFB c f) g = mapGFB c (f.g)

-- Rules to rewrite un-merged function back
"mapGGraph" [1] forall f. foldg Empty (mapGFB Vertex f) Overlay Connect = mapG f
 #-}

So, as in GHC.Base, we

Rewrite mapG into its buildG equivalent using the "mapG" rule
Try to optimize what can be optimized with the "foldg/buildG" and the "mapFB/mapFB" rules
Convert back to standard form what was not optimized using the "mapGGraph" rule

Benchmarks

Let’s verify if all of this is working. Using the great Criterion library:

import Algebra.Graph
import Control.DeepSeq
import Criterion.Main

main :: IO ()
main = defaultMain
  [ bench "foldg" $ nf foldg' $!! grInt
  , bench "foldg . mapG" $ nf (foldg' . mapG') $!! grInt
  , bench "foldg . mapG . mapG" $ nf (foldg' . mapG' . mapG') $!! grInt
  , bench "mapG" $ nf mapG' $!! grInt
  , bench "mapG . mapG" $ nf (mapG' . mapG') $!! grInt
  ]
  where
    foldg' :: Graph Int -> Int
    foldg' = foldg (0 :: Int) (const 1) (+) (+)
    mapG' :: Graph Int -> Graph Int
    mapG'  = mapG (+1)

grInt :: Graph Int
grInt = vertices [0..9999]

we will benchmark different combinations of foldg and mapG.

With rewrite rules

Compiling with $ ghc -O Test.hs -ddump-rule-firings | grep Algebra.Graph

Rule fired: mapG (Algebra.Graph)
Rule fired: foldg/buildG (Algebra.Graph)
Rule fired: mapG (Algebra.Graph)
Rule fired: mapG (Algebra.Graph)
Rule fired: foldg/buildG (Algebra.Graph)
Rule fired: mapFB/mapFB (Algebra.Graph)
Rule fired: mapG (Algebra.Graph)
Rule fired: mapG (Algebra.Graph)
Rule fired: foldg/buildG (Algebra.Graph)
Rule fired: mapFB/mapFB (Algebra.Graph)
Rule fired: foldg/buildG (Algebra.Graph)
Rule fired: foldg/mapGFB (Algebra.Graph)

So our rules are doing something… Are they doing it right ?:

benchmarking foldg
time                 96.62 μs   (96.26 μs .. 97.09 μs)
[...]

benchmarking foldg . mapG
time                 94.05 μs   (93.90 μs .. 94.22 μs)
[...]

benchmarking foldg . mapG . mapG
time                 94.57 μs   (94.39 μs .. 94.79 μs)
[...]

benchmarking mapG
time                 207.6 μs   (205.9 μs .. 209.8 μs)
[...]

benchmarking mapG . mapG
time                 201.2 μs   (199.9 μs .. 203.2 μs)
[...]

The rules are working! foldg and foldg . mapG are taking almost the same time to run. The mapG composition was also optimized away!

Without the rules

But are we just lucky ? Let’s try without the rewrite rules:

$ ghc -O Test.hs -fno-enable-rewrite-rules

benchmarking foldg
time                 95.36 μs   (95.04 μs .. 95.62 μs)
[...]

benchmarking foldg . mapG
time                 221.5 μs   (221.3 μs .. 221.8 μs)
[...]

benchmarking foldg . mapG . mapG
time                 398.3 μs   (397.4 μs .. 399.0 μs)
[...]

benchmarking mapG
time                 201.5 μs   (201.4 μs .. 201.6 μs)
[...]

benchmarking mapG . mapG
time                 348.6 μs   (348.5 μs .. 348.7 μs)
[...]

There is no luck here :) That was effectively the rewrite rules that optimizes the expressions.

Conclusion

The build technique can be used to optimize user code with many structures with a custom fold, for example with alga’s graphs, but certainly for many others data-type. Also note that almost any function based on foldg and of type Graph a -> Graph b can be optimized, not only mapG (induce and transpose composition can be optimized to same way).

GSoC 2018 results

2018-08-06T00:00:00+02:00

The project

The goal of this summer of code project was dual:

Write and use a benchmarking-suite for standard graphs libraries in Haskell.
Improve Alga’s performances.

The summary of the project can be found at https://summerofcode.withgoogle.com/projects/#5856533557018624

The beginning

During the community bonding period, I sent some patches to hash-graph and FGL while starting the benchmarking-suite. I also looked at some issues in Alga’s repository.

The community was very warm and helped me even with my newcomer questions, which is always pleasant!

The benchmarking suite

The suite is currently benchmarking 4 libraries:

With 18 functions.

Results can be found at: https://github.com/haskell-perf/graphs/tree/master/results.

The main chart:

I wrote some comments here.

I used the great criterion to benchmark time and weigh for space usage.

Functionalities

Using different parameters you can (at run-time):

Choose on which standard graphs (and their size) functions will be benchmarked.
Choose if you want to include the graph-creation time.
Choose specifically libraries/functions to be benchmarked.

It features different renderers:

In plain text
In markdown

And different analysis tools:

A summary that displays the fastest functions.
An abstract calculating a ratio between the largest benchmarks.
A quick comparison, when benchmarking two libraries, to say at a glance which one was faster.

Full usage instructions can be found in https://github.com/haskell-perf/graphs/.

Flags

The suite is easily configurable at build-time. You can disable a library (and thus not depend on it) or a functionality using cabal flags.

Chart

As requested by a reddit user, I have implemented a chart renderer using the great Chart library.

It was a hard time, because of the little error bars which are not implemented by default in the Chart library.

Alga

Tutorial

I wrote a little tutorial for newcomers to Alga. It can be found here: https://nobrakal.github.io/alga-tutorial/

Written in LaTex and converted to HTML using the great (haskell-powered) pandoc. It means that one can easily produce a PDF file out of the code or modify the text quickly without dealing with HTML.

Agda

I have made a little incursion in the proofs of the algebra of Alga formalized in agda. I didn’t know this language and I learned a lot! I only proved a little axiom and failed to prove a bigger theorem, the full story is here.

Optimizations

A big part of optimizations was made using GHC techniques like rewrite rules, specialization and inlining. I have made a summary of what I have learned here to help others. This work was mainly made in the Algebra.Graph module.

Regression suite

I have made a little script that change the benchmarking-suite into a regression suite specialized for Alga. It compares two states of the Alga repository (like master and another branch being under a pull request). It is highly tricky (in fact it is a suite of sed commands) but very useful. It can be done for any other graph library. Feel free to ask!

What is left

As always, there is some work left. Mainly I have opened some issues on Alga that I was not able to close yet. There is also some code cleanup to do in the benchmarking suite as well as in the regression suite for Alga.

If you find any missing features, don’t hesitate to tell me, I will be glad to help.

Feelings

After this list, I want to say that I had a very good time doing this Google Summer of Code. I think I learned more about functional programming during these three months than during two years in a bachelor of computer science. Moreover, the Haskell community (which I observed through reddit) is extraordinary kind.

I will be happy to continue the work started here, as I became a maintainer of alga :)

Thanks

I want to really thank Andrey Mokhov for his active and very useful mentorship! All of this project would not be what it is without his support and wise pieces of advice.

Also, thanks to the entire Haskell community for its encouragements and critics.

GHC, One compiler to RULE them all

2018-07-20T00:00:00+02:00

I am using some GHC pragmas to optimize alga and I must admit that they are very powerful. On the wise counsel of one of my GSoC mentors, A. Mokhov, here is a little summary of what I discovered, as this can help others.

GHC is the employee you need

First of all, it is important to understand the power of GHC.

As a Haskell compiler, GHC is already great, but it is greater than that because it can optimize your code.

For example, let us consider times4List that multiply by 4 every member of a list of Int.

times2Times2List :: [Int] -> [Int]
times2Times2List = map (2*) . map (2*)

Obviously, this definition is equivalent to

times4List :: [Int] -> [Int]
times4List = map (4*)

The second definition should be faster, because it traverses the list once, while the first do it 2 times.

Let us try with a very simple code to benchmark this, using Criterion:

import Criterion.Main

main :: IO ()
main = defaultMain
  [ bench "times2Times2List" $ nf times2Times2List [1..10000]
  , bench "times4List" $ nf times4List [1..10000]
  ]

times2Times2List :: [Int] -> [Int]
times2Times2List = map (2*) . map (2*)

times4List :: [Int] -> [Int]
times4List = map (4*)

If we compile the code without optimization, it produces:

benchmarking times2Times2List
time                 726.9 μs   (707.4 μs .. 749.2 μs)
[...]
benchmarking times4List
time                 397.2 μs   (396.0 μs .. 398.5 μs)
[...]

So as expected, the first is slower.

Now if we recompile the same file but asking GHC to optimize our code (with -O):

benchmarking times2Times2List
time                 215.0 μs   (214.9 μs .. 215.2 μs)
[...]
benchmarking times4List
time                 212.2 μs   (211.9 μs .. 212.6 μs)
[...]

So GHC, among other optimizations, merged the two maps.

GHC’s RULEs pragmas

But how GHC figured out to combine the two maps? With a cool pragma: the rewrites rules. It is very simple and yet very powerful.

Let us see what optimized the two first examples (this is not true, you can see the real RULES used here, but we will assume it for the sake of simplicity).

{-# RULES
  "map/map" forall f g xs. map f (map g xs) = map (f.g) xs
 #-}

Basically, this rule is telling GHC to change map f (map g xs) to map (f.g) xs when types match.

Everything is important

The actual little words “when types match” lead to something very cool: you can specialize a function to a specific type.

Imagine you have a function that internally deals with Data.Set to count the unique elements of a list:

import qualified Data.Set as Set

countUniques :: Ord a => [a] -> Int
countUniques = Set.size . Set.fromList

It is well known that Data.IntSet is faster than Data.Set. So why not use IntSet when possible? And, even better, is it possible to do the change “behind” the user’s back, so the user does not have to use the specialized version “by hand”?

module Test where

import qualified Data.Set   as Set
import qualified Data.IntSet as IntSet

countUniques :: Ord a => [a] -> Int
countUniques = Set.size . Set.fromList
{-# RULES
  "countUniques/Int" countUniques = countUniquesInt
  #-}
{-# INLINE [1] countUniques #-}

countUniquesInt :: [Int] -> Int
countUniquesInt = IntSet.size . IntSet.fromList

Now, you are telling GHC to change countUniques by the faster countUniquesInt when possible, even is the user’s code (do not pay attention to the INLINE pragma, it is discussed after).

In two other files, here is some benchmarking code:

Alone.hs

module Alone where

import qualified Data.Set as Set

countUniquesP :: Ord a => [a] -> Int
countUniquesP = Set.size . Set.fromList

Main.hs

import Test
import Alone
import Criterion.Main

aListOfInt :: [Int]
aListOfInt = [1..10000] ++ [1..1000]

main :: IO ()
main = defaultMain
        [ bench "countUniques" $ nf countUniques aListOfInt
        , bench "countUniquesP" $ nf countUniquesP aListOfInt]

Compiling with ghc Main.hs -O and running it gives:

benchmarking countUniques
time                 530.9 μs   (528.6 μs .. 533.9 μs)
[...]

benchmarking countUniquesP
time                 836.3 μs   (834.0 μs .. 838.2 μs)
[...]

Using the rewrite rule, we get more than a 50% speedup!

Try to remove the INLINE line in my previous example and re-build using ghc Main.hs -O. Ghc will complain about an INLINE pragma, and in performances:

benchmarking countUniques
time                 880.6 μs   (855.6 μs .. 907.1 μs)
[...]

benchmarking countUniquesP
time                 885.0 μs   (858.8 μs .. 910.2 μs)
[...]

The countUniques/Int rule has not fired!

So what did the INLINE pragma?

INLINE, please

Now let’s say that you are going to use a strange library which provide:

times2List :: [Int] -> [Int]
times2List = map (2*)

Then one can write:

times4List :: [Int] -> [Int]
times4List = map (2*) . times2List

So does the optimization occur?

That is the pretty part: yes it actually takes place. To help to figure out such optimizations, GHC inlines your code. Because Haskell is pure, one can replace a function by its definition.

If GHC estimate that it worth it, the compiler will replace your function by its definition. And you guessed it, GHC tries to apply RULES at each pass. So conceptually, GHC translates:

times4List :: [Int] -> [Int]
times4List = map (2*) . times2List

times4List :: [Int] -> [Int]
times4List = map (2*) . map (2*)

and apply the previous rule:

times4List :: [Int] -> [Int]
times4List = map (4*)

So GHC does this pretty well itself, but you can sometimes want to override its default behavior (in fact, it almost every time inlines small functions, but is more reluctant to inline big functions). That is what INLINE pragma is made for (and the corresponding NOINLINE).

Adding

{-# INLINE myComplicatedFunc #-}

will make GHC very keen to inline myComplicatedFunc.

Inlining phase

You can even add the phase on which you want the inlining to be done. Currently, GHC runs 3 phases (numbered from 2 to 0) in which it tries to inline and applies rewrite rules. The syntax is {-# INLINE [k] myComplicatedFunc #-}, which, quoting GHC manual, means:

do not inline myComplicatedFunc until phase k, but from phase k onwards be very keen to inline it.

Why do we care? Because of RULES! If we INLINE a function too early in the process, related RULES might not fire. So it is always a good idea to specify an INLINE pragma with phase control when using rules, to ensure that nothing is done too early preventing your super rule to fire!

So what happened to countUniques?

Because its definition is very simple, GHC inlined it and prevented the RULE to fire!

Adding {-# INLINE [1] countUniques #-} ensure that GHC will not inline countUniques too early and thus that the RULE will fire.

Why not everything is inlined?

Because it is sometimes not possible (for a recursive function for example) or sometimes useless (transforming map f xs in map (\x -> fbody x) xs is not useful) and this takes time to the compiler. Moreover inlining everything when it is not needed can lead to an enormous amount of code.

SPECIALIZE yourself

One other pragma was once a RULE but is now a pragma in full: SPECIALIZE.

When you have a class constraint in a function, for example with:

plusTwoRec :: Num a => [a] -> [a]
plusTwoRec [] = []
plusTwoRec (x:xs) = x+2:plusTwoRec xs

(It is basically map (+2) but we will not use this definition either GHC will inline it too quickly).

The function produced by GHC is containing another argument called a dictionary carrying information about class methods (here the Num class). Adding an argument and having to lookup through it takes time. So you can specialize the function using the SPECIALIZE pragma.

{-# SPECIALIZE plusTwoRec :: [Int] -> [Int] #-}

This will create another function, with the same code than the original plusTwoRec but with the specialized type (and thus, without the dictionary) and a rule to change the two functions when types match.

To benchmark all of this, we will use three files:

Alone.hs

module Alone where

plusTwoRecP :: Num a => [a] -> [a]
plusTwoRecP [] = []
plusTwoRecP (x:xs) = x+2:plusTwoRecP xs

Todo.hs

module Todo where

{-# SPECIALIZE plusTwoRec :: [Int] -> [Int] #-}
plusTwoRec :: Num a => [a] -> [a]
plusTwoRec [] = []
plusTwoRec (x:xs) = x+2:plusTwoRec xs

Main.hs

import Todo
import Alone
import Criterion.Main

aListOfInt :: [Int]
aListOfInt = [1..10000]

main :: IO ()
main = defaultMain
  [ bench "plusTwoRec" $ nf plusTwoRec aListOfInt
  , bench "plusTwoRecP" $ nf plusTwoRecP aListOfInt
  ]

(Merging Alone.hs and Todo.hs prevent the rule to fire in GHC 8.4.3, this will be corrected in a next release (see https://ghc.haskell.org/trac/ghc/ticket/15445)).

Compiling with ghc Main.hs -O and running it:

benchmarking plusTwoRec
time                 187.3 μs   (184.6 μs .. 190.8 μs)
[...]

benchmarking plusTwoRecP
time                 396.0 μs   (393.9 μs .. 398.3 μs)
[...]

Compiling with ghc Main.hs -O -fno-enable-rewrite-rules and running it:

benchmarking plusTwoRec
time                 396.3 μs   (394.4 μs .. 398.4 μs)
[...]

benchmarking plusTwoRecP
time                 395.3 μs   (393.8 μs .. 397.7 μs)
[...]

Specialization is a great way to optimize a polymorphic library when you know that it will be used mostly with one type.

Some examples of specialization can be found in the actual GHC.Base.

Why not everything is specialized?

When you have to deal with the outside you cannot tell in advance what types will your user need; moreover creating a specialized function takes space and doing this too much can create a very big library.

Inspect the core

But how to figure out what really does GHC? You can inspect the code produced after the inline/rules passes by passing -ddump-simpl to it. It will throw a lot of code, not always very understandable, but useful for comparison.

Conclusion

GHC is like your grandpa’s old car. When you are using it, it goes well, but your grandpa was able to make miracles with this old thing! How? Only because he knew how to use it properly :) .

Note: This article presents a very (very) superficial point of view about these pragmas. Please don’t hesitate to check the GHC manual related chapter or comment if you find something strange!

Benchmarking Haskell graph libraries

2018-07-05T00:00:00+02:00

The results

The main results are here https://github.com/haskell-perf/graphs/blob/master/results/ with some explanations on the benchmarks themselves.

The following of this article is some comments on these results.

One of the goals of my Google Summer of Code 2018 project was to make a benchmark table of the four main graphs libraries in Haskell:

The results are in https://github.com/haskell-perf/graphs/ where is stored the code of the benchmarking suite.

If you find a result awkward or faulty, feel free to open an issue at https://github.com/haskell-perf/graphs/issues.

Also please don’t hesitate to tweak the benchmarking suite and try yourself to produce some numbers, it is made for it!

Preliminary comment

These libraries are really different, in their inner concept, and they have to be understood before starting an analysis:

Alga is based on a new way of viewing graphs, developed by Andrey Mokhov ( thanks to him for its feedback and help concerning the benchmarks ). It is recent: the first version was uploaded on Hackage on March, 24 2017. Alga represents graph in a functional, constructively way.
Containers is based on the very fast immutable array, it implies that the bound of vertices is immutable.

But everything is immutable in Haskell !

You are right, but even if a list is immutable, you can expand a list (when adding an element). This is not true for immutable arrays. Once it is created you cannot add keys to it: in fact it already contains every vertices between the given bounds; thus you cannot add vertices as you want. It is also the only implementation of graph to support only the Int type for its vertices.

Fgl is an older try to make graph more “functional-friendly” (by Martin Erwig) with first Hackage version back in 2006 !
Hash-Graph is a try by Patrick Dougherty ( thanks to him for its feedback concerning the benchmarks ) to modernize the Fgl interface.

What to expect

In time complexities, we will note:

n for the number of vertices in the graph.
s for the algebraic size of the graph. Actually for alga’s graphs in the main results, we have s = 2 * e with e the number of edges in the graph.

So:

Alga does not assume any instance for its vertices so it cannot construct traditional efficient data-structures (like int-maps or hash-maps). This leads to a general complexity of O(s) for most of the functions (this is because of the way graphs are constructed from a list of edges. Using a more “alga” representation (like for clique) can change this complexity to O(n)).
Fgl and Hash-Graph functions are almost always requiring either an Ord or a Hashable instance for vertices, so they can build an efficient representation. This results to an average complexity of O(log(n)).
Containers is working only with Int vertices so it can build very fast representation.

That being said, lets analyze a bit the results.

Quick Analyze

Start by the end of the list

Lets starting by the end and take a look at the creation benchmark. Here there is no compromise:

Alga and Containers are the fastest.
Fgl and Hash-Graph are a bit behind.

Alga is building a “blind” representation (it does not check if you are adding an edge already here for example), so it is fast. Containers is fast thanks to its array-based implementation. The last two are building a “clever” representation, so it takes more time. The next part will give them their reward.

Inspection !

Graph inspection comprises functions like isEmpty, hasVertex, hasEdge or equality. isEmpty is efficient for everyone. Problems are coming with hasVertex:

Because Hash-Graph is representing graphs as an adjacency (hash-)map, its hasVertex works almost in constant time (real complexity of O(log n)). Fgl works with int-maps, so it has also a complexity of O(log n). Alga works in O(s). Note that hasVertex for Containers does not have any sense, since it does not make any difference between having a single vertex or not having it (its representation, an array, is a map with consecutive keys comprised between a pair of bounds).

equality is one of the big advantages of Containers which handle the question very quickly. It is one of the problems of Alga which is, behind the scene, translating its representation to adjacency maps and then comparing them.

Graph modification

Graph inspection comprise functions like addVertex, removeVertex, addEdge or removeEdge.

As it was the case for creation, Alga is faster than Fgl and Hash-Graph for adding something (for addEdge and addVertex). But because removing something implies some inspection, Hash-Graph and Fgl takes the lead for removeVertex and removeEdge.

Algorithms

One of the only comparable algorithm is dff, which produces a forest obtained from a DFS (for Deep First Search) of each vertex. Containers is built for it: it is definitely faster.

A little note about Alga which implement a transpose faster than Containers.

Others traditional algorithms are often also implemented (like BFS), but because they not produce exactly the same data-structure at the end, they cannot be compared.

Un-creation

You may want to get a list of vertices or edges in the graph, and libraries have fitting functions for this. It is graph-inspection related, so you could guess the result:

Containers is the fastest, then come Hash-Graph, close to Fgl and then Alga.

Because Hash-Graph and Fgl implement almost the same API, they have some common functions related to the theory behind. This is the case for mergeContext. No surprises, Fgl was built on it, so it is faster than Hash-Graph.

About the benchmarks

The creation time

As said in the README of results, the benchmarks are obtained by constructing and fully evaluating a graph from a list of edges, passing it to the evaluated function and then fully evaluate the results. This clearly favor libraries with a “clever” representation and does not obviously represent every use cases. Thus we provide https://github.com/haskell-perf/graphs/blob/master/results/TIME-creation.md which take into account the creation time. Alga is now becoming a serious competitor!

The representation itself

Alga is the only library for which the function that is creating a graph from a list of edges is not at all optimized; it results that the graph obtained is not very convenient to work with. Things are not the same when graphs are constructed “the alga way”. An example is taken with clique which is a very “algebraic” structure. The main comment is that it reduces the gap between Alga and the others, but not always fills it. Results can be seen here: https://github.com/haskell-perf/graphs/blob/master/results/TIME-extra.md

Conclusion

A summary: It was predictable

Due to their inner differences, Alga is faster for the creation of a graph. Fgl and Hash-Graph are better for graph inspection (the two libraries are optimized for different things, pick what you need!). Data.Graph from Containers is almost everywhere faster, but it implies to work with very specific graphs (with an immutable set of Int vertices).

Evaluate your need

There is no an “always better choice” when dealing with graphs libraries in the Haskell ecosystem. Just evaluate your needs! There is some questions that you can ask yourself:

What is the type of my vertices ? Does they have an Ord or a Hashable instance ?
I am going to modify the graph a lot ?
I am going to inspect the graph a lot ?
What standard algorithms will I need ? Are they already efficiently implemented in a library ?

Named things in Haskell

2018-05-14T00:00:00+02:00

For my GSoC project, I have to deal with named things, and I tried to generalized the traditional (String,a) object and I came with an interesting Named a data.

The problem

We often deal with named things, on which we want to perform some operations, without changing its name. For example, I want to benchmark libraries, so I will start with something like (String, Benchmark), the String being the library name, and come out to a (String, Double), with the Double being the time of taken by the function.

( For the sake of simplicity, lets imagine we have a function:

bench :: Benchmark -> IO Double

)

The Named Data

I started to write the Named data:

data Named a = Named String a

and one needed instance:

instance Show (Named a)
  show (Named n _ ) = n

And then, the main idea: applying a function f: a -> b to a Named a will produce a Named b without changing the name. My Haskell alarms ringed and detected a Functor instance:

instance Functor Named where
  fmap f (Named n x) = Named n $ f x

Well, so thought I was able to write a pretty function like:

benchN :: Named Benchmark -> IO (Named Double)

But recall, bench is of type Benchmark -> IO Double, so, using only fmap will lead to:

uglyBenchN :: Named Benchmark -> Named (IO Double)
uglyBenchN = fmap bench

So…

The Traversable instance

The solution is to provide sequence :: Monad m => Named (m a) -> m (Named a), which is inside the Traversable class. As its type says, sequence will “strip-out” the monadic effect. So we have:

instance Foldable Named where
  foldMap f (Named _ x) = f x

instance Traversable Named where
  -- Applicative f => Named (f a) -> f (Named a) 
  sequenceA (Named n m) = Named n <$> m

And poof, I am able to write:

notSoUglybenchN :: Named Benchmark -> IO (Named Double)
notSoUglybenchN = sequence . fmap bench

Because this is a common thing to do, we can use a classic function, traverse, and hence:

benchN :: Named Benchmark -> IO (Named Double)
benchN = traverse bench

Hooray ! But can we do better ? Named seems a pretty interesting object…

Is Named a Monad ? (Answer, no it is a CoMonad)

An interesting Functor is sometimes a Monad. So I started my checklist: to provide a Monad I have to provide the return and the bind function:

return :: a -> Named a
>>= :: Named a -> (a -> Named b) -> Named b

I don’t have a standard way to name things, and neither to get a name out of two named things… But I can easily strip out the name, and conserve the name:

notReturn :: Named a -> a
notBind :: Named a -> (Named a -> b) -> Named b

It seems pretty much like something dual to a monad. And this is called a Comonad :)

The instance definition:

instance Comonad Named where
  -- Named a -> a
  extract (Named _ obj) = obj
  -- Named a -> (Named a -> b) -> Named b 
  (=>>) named f = Named (show named) $ f named

Well we have a Comonad instance, we can redefine the Eq instance, given a little function:

liftExtract :: (Comonad w) => (a -> b -> c) -> w a -> w b -> c
liftExtract f a b = f (extract a) (extract b)

instance Eq a => Eq (Named a) where
  (==) = liftExtract (==)

liftExtract can in fact be used to define every lifted instances of transformers…

And can even redefine the Functor instance:

instance Functor Named where
  fmap = liftW

GSoC 2018 - The plan

2018-04-26T00:00:00+02:00

I was admitted for the Google Summer of Code 2018 with my project: Benchmarking graph libraries and optimising algebraic graphs, for the Haskell organization and under the wise supervision of Andrey Mokhov and Aloïs Cochard. So I am going to work for the next 3 months on graph libraries in the Haskell ecosystem. The work will be done in two mains repositories:

For the graph benchmarking suite: https://github.com/haskell-perf/graphs
For alga: https://github.com/snowleopard/alga And of course, regular updates will come here about the code progress :)

Alexandre Moine’s blog

Practical deforestation for fixed-point structures – introducing tungsten

Introduction

Deforestation

The Fix operator

What is this strange thing?

How the hell am I using this?

Recursion-schemes

It is cata-strophic

Catamorphisms for fixed-point structures

An example

The rule(s)

Examples

Another example

Proof

Another rule

Tests

Conclusion

Thanks

Learn-ocaml code classification

Introduction

First steps

Lambda

Applied AST fingerprinting

A cool paper

A story of sub-ASTs

Clustering

A possible semimetric

Meaning

Cluster analysis

At the end

Some results

Rev

A main class not so satisfying

A second class

The others

Conclusion

Thanks

Paramorphisms fusion for algebraic graphs

Introduction

The problem

Shortcut deforestation

Implementation for algebraic graphs

Quick reminder

hasEdge

Observations

More theory

Of bananas and barbed wire

Catamorphisms

Paramorphisms

Paramorphisms for graphs

Back to hasEdge

The goal

Is this even possible?

Implementation

Promises need to be kept

The not working but elegant version

A working solution

Foldg adaptation

Bind arguments

Warning: cycle

Conclusion

Is all of this working?

What remains to be done

Thanks

Some results about algebraic graphs homomorphisms

Prelude

Alga theory

A note on equality

Homomorphism

Graph homomorphism

Equivalence with bind-functions

Proof

Examples

Theorem 1: Composition of a graph homomorphism and a foldg function

Theorem 2: Composition of graph homomorphisms

Proof

Problems

Reduced graph homomorphism

Definition

Back to `hasEdge`