Posted by: dresstosurvive | August 18, 2007

Creating Dictionaries from Lists of Tuples in Python

Thoughts and Benchmarks.

I tested various ways of building dictionaries from lists of tuples in Python. Three ways to accomplish this are benchmarked in the code below. Not surprisingly, the default dict() constructor is the fastest. More interesting is that the functional version using map and lambda is the slowest by far. The looping version shows decent performance, but nothing special.

On a Pentium 4, 2.6ghz with 760mb of RAM under Python 2.5 on Win32 the benchmark gives these results:

$ python prof.py
map_list_to_dict took 46.063ms
loop_list_to_dict took 12.511ms
build_dict_from_list took 11.182ms

In #python @ irc.freenode.net, bpietro54 reported similar numbers on an unidentified Linux box.

$ python prof.py
map_list_to_dict took 62.500ms
loop_list_to_dict took 16.900ms
build_dict_from_list took 14.000ms

The obligatory benchmark chart (smaller bars are faster):

Benchmark Results

Unless you have a need to add tuples to a dictionary occasionally, dict() will be the fastest. In that case, adding them incrementally might be cheaper.

import time

def print_timing(func):
    def wrapper(*arg):
        t1 = time.clock()
        res = func(*arg)
        t2 = time.clock()
        return (t2-t1)*1000.0
    return wrapper

def benchmark(func, list_of_tuples):
    avg_time = [func(list_of_tuples) for i in xrange(100)]
    return sum(avg_time) / len(avg_time)

#   Dataset of 10,000 tuples
list_of_tuples = [("key_%d" % i, i) for i in xrange(10000)]

#   The messy functional approach
@print_timing
def map_list_to_dict(list_of_tuples):
    mapped_dict = {}
    map((lambda mapping: mapped_dict.update({mapping[0]: mapping[1]})), list_of_tuples)
    return mapped_dict

#   The obvious, but overcomplicated way
@print_timing
def loop_list_to_dict(list_of_tuples):
    looped_dict = {}
    for key, value in list_of_tuples:
        looped_dict[key] = value
    return looped_dict

#   Control, the normal way to do it
@print_timing
def build_dict_from_list(list_of_tuples):
    return dict(list_of_tuples)

#   The results! Hallelujah!
print "map_list_to_dict took %0.3fms" % benchmark(map_list_to_dict, list_of_tuples)
print "loop_list_to_dict took %0.3fms" % benchmark(loop_list_to_dict, list_of_tuples)
print "build_dict_from_list took %0.3fms" % benchmark(build_dict_from_list, list_of_tuples)

Responses

[...] is parsed and transformed into a dictionary using the standard dictionary constructor. You might consider another option, but this is the fastest in this [...]

Dear Master of the Obvious,

Is it entertaining to benchmark the obvious?

Leave a response

Your response:

Categories