Photos and videos are searched for duplicates based on file size and checksum. Calculated checksums are cached. On a first run, not having the cache yet, it takes 1 hour 9 minutes to process 717 GB iPhoto library located on external HDD with the 138 368 photos total.
https://github.com/senotrusov/workety - A library to run Ruby classes as daemons. Process start, signal handling, logfile/pidfile, restart by watchdog, exception handling, reporting to Airbrake and Exceptional, Rails environment load, multithreaded workers.
https://github.com/senotrusov/redis-call - A Ruby library to access Redis using Hiredis takes care of thread-aware connections, handy API for keys' names construction, transactions and queues.