Author: David

In a previous post I showed a way to calculate the total distance of a GPX track using Scala’s foldLeft. Continuing my current hobby of exploring the new Java 8 lambdas and streams API I thought I would see how the functional approach translated to Java.

To recap, a GPX track is just a sequence of latitude, longitude pairs. The first problem is that Java doesn’t have Scala’s handy tuples so we need a custom class to represent a point.

class Point {
    public final double lat;
    public final double lon;

    public Point(double lat, double lon) {
        this.lat = lat;
        this.lon = lon;
    }
}

My first thought was to use the reduce method on the Stream interface but given the slightly more complicated requirement to keep track of both the total distance and the previous point I ended up implementing a custom Collector. From the Javadoc a Collector is:

A mutable reduction operation that accumulates input elements into a mutable result container, optionally transforming the accumulated result into a final representation after all input elements have been processed. Reduction operations can be performed either sequentially or in parallel.

The Collector interface has three type parameters: T is the type of input elements to the reduction operation, A is the mutable accumulation type of the reduction operation and R is the result type of the reduction operation.

In our case, the input elements are Points and the result type is a Double (distance in miles). We just need a mutable accumulation type to keep track of the total distance and the last point in the track. Here it is:

class Result {
    private Point previousPoint;
    private double totalDistance = 0.0;
}

So we have an input type and an accumulation type. Now we can implement the Collector which uses the haversine formula to calculate great circle distance between each point in the track and put it all together:

package com.davidkeen.test;

import com.google.common.collect.ImmutableList;

import java.util.Collections;
import java.util.List;
import java.util.Set;
import java.util.function.BiConsumer;
import java.util.function.BinaryOperator;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.stream.Collector;

public class GpxDistanceCalculator {

    public static class Point {
        public final double lat;
        public final double lon;

        public Point(double lat, double lon) {
            this.lat = lat;
            this.lon = lon;
        }
    }

    private static class Result {
        private Point previousPoint;
        private double totalDistance = 0.0;
    }
    
    public static class GpxCollector implements Collector<Point, Result, Double> {

        public static double haversineDistance(Point pointA, Point pointB) {
            double deltaLat = Math.toRadians(pointB.lat - pointA.lat);
            double deltaLong = Math.toRadians(pointB.lon - pointA.lon);
            double a = Math.pow(Math.sin(deltaLat / 2), 2) + Math.cos(Math.toRadians(pointA.lat)) *
                    Math.cos(Math.toRadians(pointB.lat)) * Math.pow(Math.sin(deltaLong / 2), 2);
            double greatCircleDistance = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a));
            return 3958.761 * greatCircleDistance;
        }

        @Override
        public Supplier<Result> supplier() {
            return Result::new;
        }

        @Override
        public BiConsumer<Result, Point> accumulator() {
            return (accumulator, entry) -> {
                if (accumulator.previousPoint != null) {
                    accumulator.totalDistance += haversineDistance(accumulator.previousPoint, entry);
                }
                accumulator.previousPoint = entry;
            };
        }

        @Override
        public BinaryOperator<Result> combiner() {

            // Should not be processed in a parallel stream
            return null;
        }

        @Override
        public Function<Result, Double> finisher() {
            return accumulator -> accumulator.totalDistance;
        }

        @Override
        public Set<Characteristics> characteristics() {
            return Collections.emptySet();
        }
    }

    public static void main(String[] args) {
        List<Point> track = new ImmutableList.Builder<Point>()
                .add(new Point(51.168437004089355, -0.648922920227051))
                .add(new Point(51.16805076599121, -0.64918041229248))
                .add(new Point(51.16757869720459, -0.64995288848877))
                .build();
        double distance = track.stream().collect(new GpxCollector());
        System.out.println("Total distance: " + distance + " miles");
    }
}

So given that we need to implement a Point class and a custom Collector the end result isn’t quite as concise as Scala but it’s still a nice functional way of processing the list. Unfortunately we can’t take advantage of parallel streams as the points have to be processed in order. The more I use Java 8’s new streams and lambdas the more I like them and Collectors are a nice way of customising reduction.

 

In a previous post I gave a way of generating random test mobile numbers (Ofcom approved!) using Scala iterators.

Now that Java 8 gives us lambdas and streams I thought I would see what those generators might look like in Java.

Here’s the mobile generator:

Supplier<String> mobileSupplier = () -> String.format("447700900%03d", ThreadLocalRandom.current().nextInt(999));

Nice, still a one-liner.

Generating pseudo MAC addresses is a little bit more trouble. This supplier uses another nested stream to generate the random hex array.

Supplier<String> macSupplier = () -> ThreadLocalRandom.current().ints(6, 0, 255)
        .mapToObj(i -> String.format("%02x", i))
        .collect(Collectors.joining(":"));

You use these Supplier functions with the static generate method of Stream. Because the generators create infinite streams we use limit to just get a few values. Eg:

import java.util.concurrent.ThreadLocalRandom;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class Generators {

    private static final Supplier<String> mobileSupplier = () -> String.format("447700900%03d", ThreadLocalRandom.current().nextInt(999));

    private static final Supplier<String> macSupplier = () -> ThreadLocalRandom.current().ints(6, 0, 255)
            .mapToObj(i -> String.format("%02x", i))
            .collect(Collectors.joining(":"));

    public static void main(String[] args) {
        System.out.println("Some mobile numbers:");
        Stream.generate(Generators.mobileSupplier)
                .limit(5)
                .forEach(System.out::println);
        System.out.println();

        System.out.println("Some (pseudo) mac addresses:");
        Stream.generate(Generators.macSupplier)
                .limit(5)
                .forEach(System.out::println);
    }
}

 

Gatling is nice load testing framework that uses Scala, Akka and Netty. It’s so much better than JMeter. It’s pretty easy to get started with scenarios written in a nice Scala DSL and it produces useful reports too. It also has a Maven plugin so you can incorporate continuous performance testing in your builds.

The trouble is if you list multiple scenarios in a simulation they will all run concurrently which is probably not what you want. You need to split your tests into separate Simulation classes and run them sequentially. The plugin documentation briefly describes a way of doing this using executions but the example is a little light.

Here’s a Maven profile that when activated will run your simulations sequentially. Execute the tests with mvn test -Pperformance.

Sometimes in testing we need to generate random data. When it comes to generating mobile numbers it would be helpful if we could be sure they aren’t allocated to a real person, just in case we accidentally send 1000 text messages to random phone numbers. Luckily Ofcom has reserved ranges of numbers for dramatic use that are guaranteed not to be allocated to providers.

Here’s a little random UK mobile number generator wrapped in an infinite-length iterator:

val random = new util.Random
val randomMobile = Iterator.continually("447700900" + f"${random.nextInt(999)}%03d")

Bonus random MAC address generator

Sometimes we need MAC addresses too. This will generate strings that look like MAC addresses (6 groups of hex digits) but aren’t really. Good enough for my purposes:

val random = new util.Random
val randomMac = Iterator.continually(Array.fill[String](6)(f"${random.nextInt(255)}%02x").mkString(":"))

I wanted a way to calculate the total distance of a GPS track. A track is basically just a list of lat,long pairs – represented in Scala by the following:

Seq[(Double, Double)]

One way to do this would be to iterate over the sequence, calculating the distance between points (using the haversine formula) and updating the sum in a variable but since this is Scala we can do it in a more functional way.

According to the Scaladoc, foldLeft “Applies a binary operator to a start value and all elements of this list, going left to right.” The signature looks like this for List[A] (a List of type A):

def foldLeft[B](z: B)(f: (B, A) ⇒ B): B

The first parameter z is of type B, so it can be different from the list type. The second parameter is a function that takes a B and an A (one of the list items) and produces a B.

The neat bit is this function iterates over the list and passes in each item to function f as the value of A. For the first item, z is passed in to f as the value of B. For each subsequent item the result of the previous call to f is used.

The usual example you will find is to sum a list of integers or something similar, like this:

list.foldLeft(0)((b, a) => b+a)

This will return the sum of all the items items in a list.

My use-case was slightly different. My list consisted of tuples and I needed to apply a function to each pair of tuples in the list and accumulate the sum of those results. This meant I needed to keep track of both the current element and the previous one during the folding.

It was Matt Malone’s page of lots and lots of foldLeft examples that put me on the right track, specifically his example for Average which showed how to use a tuple as an accumulator. Here’s how I did it:

As I said, points is just a sequence of lat,long pairs, eg:

val points = List((51.168437004089355, -0.648922920227051), (51.16805076599121, -0.64918041229248), (51.16757869720459, -0.64995288848877))

We first split the list into head and tail and use head and 0.0 as initial values. The folding function then produces a Tuple2[(Double, Double), Double]. The first item of the resultant tuple is the current list element and the second item is the sum of the current total and the result of the haversineDistance function (which itself takes as input the previous element and the current element).

points match {
  case head :: tail => tail.foldLeft(head, 0.0)((accum, elem) => (elem, accum._2 + haversineDistance(accum._1, elem)))._2
  case Nil => 0.0
}

The pattern matching is used to handle the case of an empty list.

Phew! This one line took me a while to figure out but it packs a lot of work into just a single line of code and shows how powerful Scala can be. Experimenting with the REPL was invaluable here.

For completeness, here’s the haversine function which calculates the distance between two points on the Earth (3958.761 is the mean radius of the Earth in miles):

def haversineDistance(pointA: (Double, Double), pointB: (Double, Double)): Double = {
  val deltaLat = math.toRadians(pointB._1 - pointA._1)
  val deltaLong = math.toRadians(pointB._2 - pointA._2)
  val a = math.pow(math.sin(deltaLat / 2), 2) + math.cos(math.toRadians(pointA._1)) * math.cos(math.toRadians(pointB._1)) * math.pow(math.sin(deltaLong / 2), 2)
  val greatCircleDistance = 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a))
  3958.761 * greatCircleDistance
}