Low Hanging Data is a project built on a simple conviction: the best data analysis is the simplest analysis that answers the question. Before reaching for a model, a complex pipeline, or an elaborate visualization, pick the fruit that's already within reach.

About the Author

Michael Petrillo is a data engineer, data scientist, and data analyst with a focus on data operations. He writes about practical data workflows, pipeline architecture, and applied analysis techniques that prioritize clarity over complexity.

His 2020 master's thesis at Colorado State University Global, Stock Change Prediction Utilizing Social Media Pools (PDF), tested whether Reddit comment data improves a support vector machine's short-term stock predictions. The project write-up covers what it found and what a rebuild should change.

Connect with Michael on LinkedIn

The Three Principles

• Concise — Good analysis fits in a headline with supporting evidence. If a finding requires three paragraphs to explain, it either isn't clear yet or wasn't worth finding. See: Visualizing Data with Python, Organizing Data with SQL.
• Transparent — Sources, assumptions, and limitations are stated plainly. Reproducibility is not optional. If someone else can't verify the work, the work isn't finished. See: Data Cleaning and Validation, Building Your First Data Pipeline.
• Low Hanging Fruit First — Counts, sums, averages, and distributions come before clustering, forecasting, or causal inference. Complexity is only added when it changes a conclusion. See: Getting Started with Data Collection, DuckDB for Financial Analysis.

Who This Is For

Anyone who works with data and wants to produce analysis that decision-makers actually trust and use:

• Analysts who want their work to have more impact with less overhead.
• Developers building data features who need to communicate findings to non-technical stakeholders.
• Researchers who want their methodology to be reproducible and their conclusions defensible.
• Anyone who has ever been asked "so what does this actually mean?" after presenting data.

Articles

All 50 articles, grouped by primary topic, newest first. Each one leads with the simplest viable approach.

Collection

• Start With Data Stakeholders Already Trust
• The Golden Age of API Access Is Over
• Working with WebSockets and Streaming Data
• Low-Hanging Data Sources for Stock Market Prediction
• Pulling Data from REST APIs
• Working with CSV and JSON
• Getting Started with Data Collection

Preparation

• Excel to SQL: Low Hanging Fruit for Making the Switch
• Python Virtual Environments
• Working with Parquet and DuckDB
• Data Cleaning and Validation
• Python & Pandas for Data Wrangling
• Organizing Data with SQL

Pipelines

• Bronze, Silver, Gold: The Medallion Architecture Explained
• Building an Agent Harness for Data Engineering
• Operational Telemetry, Explained for the Person Reading the Dashboard
• How to Build a Data Pipeline
• Stop Forcing Tools Into Jobs They Weren't Built For
• The Silent Death of Orphan Data Pipelines
• ETL vs ELT: Choosing the Right Pipeline Pattern
• Change Data Capture Requires an ROI to Be Taken Seriously
• Scheduling and Automating Data Pipelines
• Building Your First Data Pipeline

Analysis

• Call Your Shot: Feedback Loops, Experiments, and Writing It Down First
• Physics Has Models. Machine Learning Has Black Boxes.
• Think With Data. Rest to Create.
• Ontology vs Semantic Layer: What Each One Actually Is
• Analytics Paints the Picture. It Does Not Prove the Story.
• Dashboards Are Waiting Rooms: Interconnectivity Is the Endgame
• Don't Build Analytical Castles on Sand
• The Gas Gauge Is the Hardest Chart to Build
• Building a Stock Prediction Classifier with scikit-learn
• DuckDB for Financial Data Analysis
• Visualizing Data with Python

Culture & Communication

• How to Start Maintaining an Ontology as a Non-Technical Business Owner
• How to Be a Data Champion
• The Telephone Game Is How Analytics Goes Wrong
• Trust and Time Are the Real Currencies of Data ROI
• Low Hanging Fruit Reduces Risk and Builds the Expertise to Climb Higher
• Fear the Black Box: Why Data Must Be Understood End to End
• Leave the Ivory Castle: How SMEs Expose the Gaps Your Data Hides
• KPIs Are a Cultural Change, Not a Dashboard Project
• The Data Landscape Has Expanded — And So Has Its Audience
• Crawl, Walk, Run: Why Many Attempts Beat One Perfect Try
• Write for the Executive. Survive the Analyst.
• Not Everyone Is a Data Analyst (And Your Deliverables Should Reflect That)

Career

• From Cheerleader to Quarterback: Why Data Professionals Must Be Half Subject Matter Expert
• Data Careers Are Not Pokemon Evolutions

Project Writeups

• Building a Stock Prediction Pipeline: What We Did and What We Learned
• How BD Used AWS to Stop Guessing When Medical Devices Would Fail

Contact

Have a question, found an error, or want to discuss something from an article? Reach out by email:

admin@lowhangingdata.com

Legal

• Privacy Policy — how this site handles cookies and advertising data
• Terms of Use — content license, disclaimers, and site policies