Role

Back-end Infrastructure and Data Engineer

Stack

MySQL, Hadoop

MySQL to Hadoop Loader- Lookout

Wrote MySQL to Hadoop loader (similar to SQOOP) for firewall heavy infrastructures at Lookout Mobile Security

  • Wrote and tuned a MySQL DB loader that loaded data in Hadoop by breaking the database in to chunks
  • Added ability to repeat-fetch on errors and load a single chunk at a time