Hadoop Developer

Amulya kasaraneni | 816-328-6356 | amulya.hdev@gmail.com

SUMMARY:
 Around 6 years of IT experience in Analysis, design, development, implementation, maintenance and support
with experience in developing strategic methods for deploying big data technologies to efficiently solve Big Data
processing requirement.
 Around 4 years of experience on Big data using Hadoop ecosystem and related technologies such as HDFS, Map
Reduce, Spark, Hive, Pig, HBase, Oozie, Zookeeper, Sqoop, Flume and Apache Kafka.
 Excellent understanding /knowledge on Hadoop and various components such as HDFS, Job Tracker, Task
Tracker, Name Node, Data Node, Resource Manager (YARN).
 Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems (RDBMS)
and from RDBMS to HDFS.
 Experience in working with data ingestion tools like Apache NiFi and real time data transfer tool Apache Kafka
and implemented both producer and consumer configurations.
 Expertise in loading both structured and semi structured data into Snowflake.
 In-depth work experience with various file formats like XML, Json, Avro, Parquet, Sequence files, Tab delimited
files, text files, ORC files and different types of compression techniques snappy, gzip, lz4.
 Experience in data analysis using HIVE, Pig Latin, HBase and custom Map Reduce programs in Java.
 Hands on experience with Scala for both batch processing and spark streaming data.
 Experience in developing jobs using Spark framework modules like Spark-Core, Spark-SQL and Spark-Streaming.
 Experience in managing and reviewing Hadoop log files.
 Excellent understanding and knowledge of NOSQL databases like HBase, Cassandra.
 Implemented Hadoop based data warehouses, integrated Hadoop with Enterprise Data Warehouse systems.
 Expertise in moving data from Internal stage to Snowflake.
 Experience in migrating data from other sources to Snowflake data warehousing system
 Involved in Regular Pi Planning meetings.
 Good experience working with Horton works Distribution, MapR and Cloudera Distribution.
 Experience in Object Oriented Analysis Design (OOAD) and development of software using UML Methodology,
good knowledge of J2EE design patterns and Core Java design patterns.
 Experience in writing UNIX/LINUX shell scripts to automate the validation procedures.
 Experience in all stages of SDLC (Agile, Waterfall), writing Technical Design document, Development, Testing and
Implementation of Enterprise level Data mart and Data warehouses.

Delivery Assurance - Quality Focused & Process Oriented:

 Ability to work in high-pressure environments delivering to and managing stakeholder expectations  
 Application of structured methods to: Project Scoping and Planning, risks, issues, schedules and deliverables.
 Strong analytical and Problem-solving skills.
 Good Inter personnel skills and ability to work as part of a team. Exceptional ability to learn and master new
technologies and to deliver outputs in short deadlines.

TECHNICAL SKILLS:

Big Data Ecosystem Apache Sqoop, Apache Flume, Apache Kafka, HDFS, Map Reduce, Spark,
Hive, Pig, HBase, Oozie, Zookeeper.
DBMS/Databases MySQL, SQL Server, Oracle DB
NOSQL Databases HBase, Cassandra WORK
Programming Languages Java, Python, Scala, XML EXPERIENCE
Operating Systems Windows Vista/XP/NT/2000/ LINUX (Ubuntu, Cent OS), UNIX
Methodologies
Role HadoopAgile, Waterfall
Developer Start Date Jan 2019
Version
Client Control Tools SVN, GitHub
Starbucks, Seattle, WA End Date Till Date
Responsibilities:
 Handled data coming from different sources and involved in HDFS maintenance and loading of structured and
unstructured data.
 Mainly Involved in handling of large transactional data and do process and loaded in HDFS.
 Fetching cornerstone data through hive & used for application in different business scenario as we can be
considering source data.
 Involved in creating Hive Tables, loading the data from cornerstone.
 Mainly involved in data ingestion to handle the files from source and process it and load In hive tables.
 Developed HIVE queries for the analysts & processed data is imported into Hive warehouse which enabled
business analysts and operation groups to write Hive queries.
 Experience in developing Hive queries which will invoke and run Map Reduce jobs in the backend.
 Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
 Created staging (landing zone) and final tables loads.
 Involved in data extensive data validation and data comparison between files with landing tables and landing
tables vs final tables
 Expertise in identifying and solving the issues caused while loading data into Snowflake 
 Working with the client and business management on gathering requirements and understanding functional
aspects of the application.
 Good knowledge in snowflake for implementing the logics using SQL  
 Coordinates with other team members to ensure that all work products integrate as a complete solution and
adopts a supporting role to any other team member to resolve issues, or to complete tasks sooner.
 Worked on Scala programming in developing spark streaming jobs for building stream data platform integrating
with Kafka.
 Performed analytics on top of that processed data using SparkSQL
 Worked on copying of data between clusters.
 Involved in Agile SDLC during the development of project.
 Used Amex Internal framework Event engine to trigger the jobs and monitor the jobs.
 Used Rally for project tracking and project status.

Environment: MYSQL, HDFS, Hive, Scala, Spark Cornerstone, SparkSQL, Oozie, Event Engine, UNIX, Shell Scripting, Oracle
11g/12g, Apache Kafka, Snowflake.

Role Hadoop Developer Start Date Feb 2018

Client Amerisource Bergen, Chesterbrook, PA End Date Jan 2019

Responsibilities:
 Imported data using Sqoop to load data from MySQL to HDFS and Hive and vice versa.
 Involved in loading data from UNIX file system to HDFS.
 Creating Hive tables and working on them using Hive QL.
 Involved in creating Hive tables, loading data and writing hive queries which will internally run Map reduce Jobs.
 Experience in installing configuring and using Hadoop ecosystem components.
 Accessed the data into Spark from HDFS and done schema validations with help of Scala scripts.
 Performed Batch processing on data present in the Hadoop clusters and hive data as well by accessing data with
the help of Hive Context.
 Experience in loading data from Hive/SQL and performing spark transformations using Spark RDDs and Spark
data frames.
 Experienced in managing and reviewing Hadoop log files and monitored the workflow and troubleshoot the
problems encountered in job execution.
 Implemented Zookeeper configurations for better cluster coordination services in the ETL flow.
 Developing Scripts and Batch Job to schedule various Hadoop Program.
 Involved in Requirement Gathering to connect with BA.
  Working Closely with BA & vendor for creating technical Documents and Design specifications.
 Writing Hive queries to read from HBase.
 Serializing JSON data and storing the data into tables using Hive.
 Hive Data sampling, Buckets and Cluster methods for schema.
 Writing the Hadoop Job workflows & scheduling using Oozie.
 Good experience in Oozie Framework and Automating daily import jobs.
 Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access.
 Created an e-mail notification service upon completion of job for the team which requested for the da
 Developed Hive queries to process the data for visualizing in supporting with BA team.
 Involved in HP Application life cycle Management and JIRA - Agile methodology for task distribution with
estimates.

Environment: Hadoop Framework, Map Reduce, Spark, Hive, Sqoop, Pig, HBase, Flume, Oozie, Java (JDK1.8), UNIX Shell
Scripting, Oracle 11g/12g, Windows NT, Apache NIFI

Role Hadoop Developer Start Date Nov 2016

Client ICICI Bank, Hyderabad, India End Date Jun 2017

Responsibilities:
 Responsible for building scalable distributed data solutions using Hadoop Ecosystem.
 Responsible for writing MapReduce jobs to handle files in multiple formats (JSON, Text, XML etc.)
 Ingested the data in real time using Apache Kafka into spark streaming objects and implemented schema
externally using Struct Types and case classes.
 Implemented Scala functions to aggregate the data in the Hadoop clusters by accessing the data into Spark.
 Developed PIG UDFs to perform data cleansing and transforming for ETL activities.
 Generating the daily, bimonthly reports from HBase table by using Hive Serde Properties.
 Experienced in updating HBase column values based on the TimeToLive (TTL).
 Developed HIVE UDF, UDAF and UDTF for Data analysis and Hive table loads.
 Extensively worked on Production Environment for Horton works Distribution.
 Worked extensively on creating combiners, Partitioning, Distributed cache to improve the performance of
MapReduce jobs.
 Worked on Creating the MapReduce jobs to parse the raw web logs data into delimited records.
 Used Pig to do data transformations, event joins and some pre-aggregations before storing the data on the
HDFS.
 Involved in loading and transforming large sets of structured, semi structured and unstructured data.
 Responsible for creating bucketing and partitioning tables on the customer data to get the query optimization.
 Created partitioned tables in Hive for best performance and faster querying
 Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
 Developed Pig Scripts to pull data from HDFS.
 Consumed XML messages using Kafka & processed xml using Spark Streaming to capture UI updates.

Environment: Hadoop Framework, MapReduce, Hive, Sqoop, Pig, HBase, Flume, Oozie, Kafka, Spark Java, UNIX Shell
Scripting, Oracle 11g/12g, Windows NT, TOAD 9.6.

Role Java Developer Start Date Aug 2014

Client Maxim Solutions, Chennai, India End Date Nov 2016

Responsibilities:
 Developed Servlets and JSP based on MVC pattern using Struts Action framework.
 Used Tiles for setting the header, footer and navigation and Apache Validator Framework for Form validation.
 Parsing high-level design spec to simple ETL coding and mapping standards.
  Used Log4J logging framework to write Log messages with various levels.
 Involved in fixing bugs and minor enhancements for the front-end modules.
 Used JUnit framework for writing Test Classes.
 Coded various classes for Business Logic Implementation.
 Develop and test the code according to the requirements.
 Preparing and executing Unit test cases
 Doing functional and technical reviews
 Support to the testing team for System testing/Integration/UAT.
 Assuring quality in the deliverables.
 Conducted Design reviews and Technical reviews with other project stakeholders.
 Implemented Services using Core Java.
 Developed and deployed UI layer logics of sites using JSP.
 Struts (MVC) is used for implementation of business model logic. Worked with Struts MVC objects like Action
Servlets, Controllers, and Validators, Web Application Context, Handler Mapping, Message Resource Bundles
and JNDI for look-up for J2EE components.
 Developed dynamic JSP pages with Struts.
 Developed analyze and writing sql queries.
 Used built-in/custom Interceptors and Validators of Struts.
 Involved in the complete life cycle of the project from the requirements to the production support.
 Worked with JavaScript calls as the Search is triggered through JS calls when a Search key is entered in the
Search window
 Worked on analyzing other Search engines to make use of best practices.
 Collaborated with the Business team to fix defects.
 Worked on XML, XSL and XHTML files.
 Used Ivy for dependency management.
 As part of the team to develop and maintain an advanced search engine, would be able to attain expertise on a
variety of new software technologies.

Environment: J2EE, JDBC, Java 1.4, Servlets, JSP, Struts, Hibernate, Web services, SOAP, MVC, HTML, JavaScript 1.2,
WebLogic 8.0, XML, JUnit, Oracle 10g, PL/SQL My Eclipse.