Samplesort is a sorting algorithm that is a divide and conquer algorithm often used in parallel processing systems. Conventional divide and conquer sorting algorithms partitions the array into sub-intervals or buckets. The buckets are then sorted individually and then concatenated together. However, if the array is non-uniformly distributed, the performance of these sorting algorithms can be significantly throttled. Samplesort addresses this issue by selecting a sample of size from the -element sequence, and determining the range of the buckets by sorting the sample and choosing elements from the result. These elements (called splitters) then divide the array into approximately equal-sized buckets. Samplesort is described in the 1970 paper, "Samplesort: A Sampling Approach to Minimal Storage Tree Sorting", by W. D. Frazer and A. C. McKellar.

Algorithm

Samplesort is a generalization of quicksort. Where quicksort partitions its input into two parts at each step, based on a single value called the pivot, samplesort instead takes a larger sample from its input and divides its data into buckets accordingly. Like quicksort, it then recursively sorts the buckets.

To devise a samplesort implementation, one needs to decide on the number of buckets . When this is done, the actual algorithm operates in three phases:

1. Sample elements from the input (the splitters). Sort these; each pair of adjacent splitters then defines a bucket.

2. Loop over the data, placing each element in the appropriate bucket. (This may mean: send it to a processor, in a multiprocessor system.)

3. Sort each of the buckets.

The full sorted output is the concatenation of the buckets.

A common strategy is to set equal to the number of processors available. The data is then distributed among the processors, which perform the sorting of buckets using some other, sequential, sorting algorithm.

Pseudocode

The following listing shows the above-mentioned three step algorithm as pseudocode and shows how the algorithm works in principle. In the following, is the unsorted data, is the oversampling factor, discussed later, and is the number of splitters.

function sampleSort(A[1..n], k, p)
    // if average bucket size is below a threshold switch to e.g. quicksort
    if n / k < threshold then smallSort(A) 
    /* Step 1 */
    select S = [S1, ..., S(p−1)k] randomly from // select samples
    sort S // sort sample
    [s0, s1, ..., sp−1, sp] <- [-∞, Sk, S2k, ..., S(p−1)k, ∞] // select splitters
    /* Step 2 */
    for each a in A
        find j such that sj−1 < a <= sj
        place a in bucket bj
    /* Step 3 and concatenation */
    return concatenate(sampleSort(b1), ..., sampleSort(bk))

The pseudo code is different from the original Frazer and McKellar algorithm. In the pseudo code, samplesort is called recursively. Frazer and McKellar called samplesort just once and used quicksort in all following iterations.

Complexity

The complexity, given in Big O notation processors:

Find the splitters.

Send to buckets.

1. for reading all nodes

2. for broadcasting

3. for binary search for all keys

4. to send keys to bucket

Sort buckets.

1. is the complexity of the underlying sequential sorting method..

for big input sequences. In experiments, conducted by Frazer and McKellar, the algorithm needed 15% fewer comparisons than quicksort.

Sampling the data

The data may be sampled through different methods. Some methods include:

1. Pick evenly spaced samples.

2. Pick randomly selected samples.

Oversampling

The oversampling elements. In this case, no bucket takes longer to sort than the others, because all buckets are of equal size.

splitters randomly.

Bucket size estimate

.

, of which a maximum of

Using the Chernoff bound

Many identical keys

times are likely to become pivots.

Uses in parallel systems

Samplesort is often used in parallel systems, including distributed systems such as bulk synchronous parallel machines. Due to the variable amount of splitters (in contrast to only one pivot in Quicksort), Samplesort is very well suited and intuitive for parallelization and scaling. Furthermore, Samplesort is also more cache-efficient than implementations of e.g. quicksort.

. Since the buckets are sorted concurrently, the processors will complete the sorting at approximately the same time, thus not having a processor wait for others.

On distributed systems processors.

with binary search.

Experiments performed in the early 1990s on Connection Machine supercomputers showed samplesort to be particularly good at sorting large datasets on these machines, because its incurs little interprocessor communication overhead. On latter-day GPUs, the algorithm may be less effective than its alternatives.

Efficient Implementation of Samplesort

As described above, the samplesort algorithm splits the elements according to the selected splitters. An efficient implementation strategy is proposed in the paper "Super Scalar Sample Sort". (the original array containing the input data and a temporary one) for an efficient implementation. Hence, this version of the implementation is not an in-place algorithm.

In each recursion step, the data gets copied to the other array in a partitioned fashion. If the data is in the temporary array in the last recursion step, then the data is copied back to the original array.

Determining buckets

time for each element.

Super Scalar Sample Sort uses a balanced search tree which is implicitly stored in an array . Given the search tree , the algorithm calculates the bucket number evaluates to 1 if it is true and 0 otherwise):

j := 1
repeat log2(p) times
    j := 2j + (a > tj)
j := j − p + 1

Since the number of buckets is known at compile time, this loop can be unrolled by the compiler. The comparison operation is implemented with predicated instructions. Thus, there occur no branch mispredictions, which would slow down the comparison operation significantly.

Partitioning

For an efficient partitioning of the elements, the algorithm needs to know the sizes of the buckets in advance. To partition the elements of the sequence and put them into the array, we need to know the size of the buckets in advance. A naive algorithm could count the number of elements of each bucket. Then the elements could be inserted to the other array at the right place. Using this, one has to determine the bucket for each elements twice (one time for counting the number of elements in a bucket, and one time for inserting them).

bits, these cost are small compared to the space of the input array.

In-place samplesort

A key disadvantage of the efficient Samplesort implementation shown above is that it is not in-place and requires a second temporary array of the same size as the input sequence during sorting. Efficient implementations of e.g. quicksort are in-place and thus more space efficient. However, Samplesort can be implemented in-place as well.

The in-place algorithm is separated into four phases:

1. Sampling which is equivalent to the sampling in the above-mentioned efficient implementation.

2. Local Classification on each processor, which groups the input into blocks such that all elements in each block belong to the same bucket, but buckets are not necessarily continuous in memory.

3. Block permutation brings the blocks into the globally correct order.

4. Cleanup moves some elements on the edges of the buckets.

One obvious disadvantage of this algorithm is that it reads and writes every element twice, once in the classification phase and once in the block permutation phase. However, the algorithm performs up to three times faster than other state of the art in-place competitors and up to 1.5 times faster than other state of the art sequential competitors. As sampling was already discussed above, the three later stages will be further detailed in the following.

Local classification

buffers that are of equal size to the blocks, one for each bucket. Thereafter, each processor scans its stripe and moves the elements into the buffer of the according bucket. If a buffer is full, the buffer is written into the processors stripe, beginning at the front. There is always at least one buffer size of empty memory, because for a buffer to be written (i.e. buffer is full), at least a whole buffer size of elements more than elements written back had to be scanned. Thus, every full block contains elements of the same bucket. While scanning, the size of each bucket is kept track of.

Block permutation

subarray for each bucket.

, the swap buffers are empty again. Otherwise the block remaining in the swap buffers has to be inserted into its destination bucket.

If all blocks in the subarray of the primary bucket of a processor are in the correct bucket, the next bucket is chosen as the primary bucket. If a processor chose all buckets as primary bucket once, the processor is finished.

Cleanup

Since only whole blocks were moved in the block permutation phase, some elements might still be incorrectly placed around the bucket boundaries. Since there has to be enough space in the array for each element, those incorrectly placed elements can be moved to empty spaces from left to right, lastly considering the overflow buffer.

See also

• Flashsort

• Quicksort

References

External links

Frazer and McKellar's samplesort and derivatives:

• Frazer and McKellar's original paper

• DOI.org

• DOI.org

Adapted for use on parallel computers:

• http://citeseer.ist.psu.edu/91922.html

• http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.214