Cloud/Data Engineer
As a Cloud/Data Engineer you can say that I'm a
data-driven musician. Because data is everywhere.
I'm a Data guy that likes to do stuff in the cloud, and an Electrical Engineer with Project Management. And a musician. In the end, everything is data, so feel free to explore my portofolio. Here you will find links for my portfolio, certificates, blog posts about career transitioning, professional experiences and random perks that you might find interesting.
I have a diploma in Electrical Engineer and Project Management. With that, I've been over 7 years managing people, budget, schedule and gathering/reporting data on electrical substations. Music was always a hobby and even with that I collect data to leverage my skills on the instrument.
This is my experience in the Data Engineering field, including experience as an Electrical Engineer and Project Manager.
This section shows all the certificates that I gathered and courses that I completed to hone my data skills.
Maybe the biggest certification for Data Engineers on the cloud, specially the beta one! This was by far the hardest test I have done in my life, everything was doubled in size, 120 questions and 4 hours of test (60 and 2 hours for the normal test) and no available courses for the updated content, very unfair to your mind since you can't go to the bathroom during the test. I learned the beauty of studying through documentation.
I'm really fond of Google Cloud Platform tools. For me, they provide a more clean and effective suite of resources that you can use more efficiently. It was really fun getting this certificate.
My first great achievement in the cloud. It showcases knowledge and skills in AWS technology, across a wide range of AWS services.
It was free, so I got it.
This exam assesses your knowledge of designing and creating DAGs as reliable data pipelines following best practices.
I learned many tools for Data Engineering: SQL, RDBMS, ETL, data warehousing, NoSQL, bash commands, big data and Spark, Airflow and Kafka
In the eight courses I have learned how to use spreadsheets, SQL, Tableau, R and a capstone project that you can check on my projects.
An introduction to programming using Python, a popular language for general-purpose programming, data science, web programming, and more.
I learned SQL for data wrangling, Spark SQL, AB testing and a capstone project that you can check on my projects.
Yes. I also have a music certification. The course explores different types and styles of compositions from the perspective of the improviser.
Here are my formal education and diplomas.
This section shows all my projects in Data Engineering and Data Analysis so far. You can find them hosted on GitHub and Kaggle.
I'm not very proud of this one. I usually do these projects with open-source tools for my post-graduation assignments. This assignment was not a project per se, but rather a step-by-step activity that I had to finish. Anyway, at least I can say that I know MongoDB.
I found a very interesing dataset, the Portal da TrasparĂŞncia (Transparency Portal). I learned a lot about API calls and more advance data cleaning using Spark. Of course, I learned a lot of Cassandra too.
Yeah, I know it's getting repetitive, I have to find another great dataset for my labs. But at least I learned more Spark! I even grasped some machine learning in this one! It's very rudimentary, but you get the point.
Apache Hadoop might be the most iconic tool used in Data Egineering to this date, if we could define a tool that would define big data, it would be it. Very interesting to use Hive on top of the MapReduce concept, it proves to be a very efficient way to analyse large amounts of data.
Data pipeline using Azure, Databricks and Power Bi. It's really amazing how they converse well, it's very easy to access Databricks through Azure.
This is a challenge I have made for the role of Data Engineer. I had to clean and upload six .csv files to a database in the cloud, make some data analysis and some visualizations. Of course I used AWS, I also made and ERD to make my schema creation easier, used Lambda for the ETL process, Pandas for EDA and data cleaning and QuickSight for the data visualizations. Of course there is more so do checkout the project.
This is an excelent project to hone your AWS and cloud skills, since all cloud counterparts (GCP, Azure, etc) use similar structure compared to this. You will learn how to use VPCs, subnets, security groups, NAT gateways, route tables, VMs, file systems, databases, create domain names, SSL certificates and IAM roles, making it a great add to your portfolio.
This is an ETL project using the Spotify API to get the daily top 50 global songs in the app. I used AWS to extract the file from a Jupyter notebook using Python to process into an S3 bucket, separating data between artist, album and songs into another S3 bucket, loading the data into AWS Athena for analysis using AWS Glue.
I used BigQuery, Excel and Tableau to use the Google Data Analytics motto: Ask, Prepare, Process, Analyise, Share and Act.
I usesd Data Wrangling and Data Analysis skills with SQL deployed in Python using Pandas, Numpy and Matplotlib by creating a database with Data Governance concepts.
I made an ETL (Extract, Transform and Load) pipeline, commonly used in Data Engineering to extract the data from two json files that has a list of banks by market cap, transform them by multiplying it to a csv file that has the rates of all currencies referenced in USD, loading it into a simple csv file.
Here are my Medium posts about my career transition from Electrical Engineering do Analytics Engineering.
This post is more of a personal guide for this project, showing the importance of knowledge acquired and my pain in the process of deploying the resources of this project..
This post shows the advantages and skills that you can transfer from Electrical do Data on the cloud.
This post shows the hardships on the transition from Electrical do Data Engineering.
I'm a passionate data guy and eager to find new people that share this sentiment so feel free to contact me anytime. We could even insert some music in the context, why not?