04Area of Expertise

Data Platforms & Automation

One number instead of three.

In most companies the data already exists - in the ERP and the accounting system, in exports one person turns into a spreadsheet every month, in Excel files that have quietly become databases, sometimes on paper in the warehouse. Some of it doesn't exist yet: it has to be collected from outside the company, or recorded for the first time where the work happens. We build the layer between those sources and your decisions: pipelines that collect and reconcile the data, a data warehouse or lakehouse that keeps it, dashboards that show it, and the automation that runs all of it unattended. We choose the platform by the requirement - volume, where the data may live, who operates it - and we've built the same thing on a client's own server and in Azure with equally good reasons. If what you need is the system that produces the data in the first place - an application, a portal, an internal tool - that lives under Application and Systems Engineering; here we start where that data has to be brought together.

At a glance
  • We turn what you already have - ERP and accounting exports, Excel and CSV files, databases, APIs, and a connector we build where there is no other way - into scheduled, monitored pipelines with a documented warehouse behind them.
  • We choose the platform by how much data you have and what you already run - from a single PostgreSQL database to Databricks, Snowflake, or BigQuery if you are already there. We have no one product we sell to everyone.
  • Power BI as the reporting default - automated refresh, with the data staying in your own Microsoft 365 tenant and on your own server.
  • Every pipeline validates before it writes, and every run reports the same day.
  • Work done by hand on a schedule - merging exports, re-keying figures, checking totals - becomes a scripted, monitored job, in Python, PowerShell, or Azure Functions; where the figures aren't recorded at all today, we start by giving someone something to record them with.
How we choose what fits

Before we recommend a platform, we answer seven questions:

  1. 01

    The decision - what the data is for and who uses it: people through dashboards, or other systems through an API. That decides what we collect and how often.

  2. 02

    The sources - ERP and accounting systems, exports and spreadsheets, databases, APIs, and, where a system offers nothing else, a connector we build; who is responsible for each one - your team, somebody outside, or nobody - and what happens when a source changes without warning.

  3. 03

    The manual step - what somebody does by hand right now: re-keys figures, merges files, checks totals by eye; and what happens when that person is on holiday.

  4. 04

    History - today's snapshot or the trend over time; spreadsheets rarely keep history, and the answer changes the whole schema.

  5. 05

    Volume - thousands of rows a day, or millions; distributed processing when the case demands it.

  6. 06

    Where the data may live - your server, your cloud tenant, a specific region. For personal data and your customers' data, residency and contracts decide before performance does; for public data they do not decide at all, and we say so straight away.

  7. 07

    Who runs it day to day - your IT, an analyst, or us.

Most SME data problems aren't volume problems; they're consolidation and trust problems. A figure two systems disagree on is a figure nobody acts on, and that argument is settled not by the size of the platform but by a definition people have agreed to.

What we build

Data warehouses and lakehouses

  • PostgreSQL - a robust base for most warehouses: on your own server or an Azure VM, with history kept at the level of detail the question needs - daily snapshots, event records, or both. It carries far more data than most SMEs will ever produce.
  • Microsoft Fabric or Azure SQL - when the organization already lives in Microsoft 365 and wants a managed platform with Power BI built in; Synapse where it's already in place.
  • Databricks (PySpark, Delta Lake) - when volume, variety, or analytical workload outgrows a single database; notebooks for analysts on the same governed data.
  • Redshift, Snowflake, BigQuery - when you're already on that platform. We design for the platform you have before proposing another.

A few thousand rows a day is not big data. A cluster bought for a workload that fits in PostgreSQL is money and complexity you'll carry for years.

Integration and pipelines

  • Files and exports - CSV and Excel exports from your ERP, accounting, or warehouse system, picked up on a schedule, validated, and loaded - instead of one person merging tabs by hand.
  • Systems, databases, and APIs - direct connections to the systems you already use, with the data pulled incrementally on a schedule.
  • Python pipelines or Azure Data Factory - scheduled Python jobs as our default, validated before every write and exiting with a status a scheduler can act on; Data Factory when the organization wants managed, low-code orchestration inside its Azure tenant.
  • Paper and manual records - a simple capture form or barcode scan at the point of work, so nobody re-keys from paper later.

A pipeline that wakes nobody when it stops is worse than the file somebody made by hand - at least somebody noticed when that one was missing.

Reporting and analysis

  • Power BI - dashboards built around the decisions your team actually makes, refreshed automatically; through the on-premises gateway into your own tenant when the database runs on your server, natively in Fabric when the data already lives in the cloud. If you can't name the decision a dashboard serves and the person who makes it, we don't build that dashboard.
  • Databricks notebooks and SQL - when analysts need to ask questions a dashboard can't answer, on the same data the dashboards use.
  • AI-ready foundation - clean, historical, governed data is what any AI initiative needs first: without history measured in months rather than days, and without field definitions that agree across sources, there is nothing to build on. We build that, and we say plainly what your data is ready for - including when the answer is that it isn't ready for anything yet.

A dashboard on ungoverned data automates the wrong answer. Quality gates go in front of the warehouse.

Quality and governance

  • Validation before writing - the expected row count, required fields, plausible values; the checks a person does by eye today become rules that run every time, and a run that falls outside the validation thresholds leaves the database untouched and a file for whoever owns that source to correct - your side or ours, agreed before we build.
  • Definitions and history - every field means one thing, written down; raw source records kept beside cleaned ones, so we can show what was known on a given day.
  • Personal data - protection built into the schema: fields stripped before they're saved, records that are never stored at all, retention that's decided, not accidental.

Governance bought as a product before it's designed as a practice ends up as a catalog nobody fills in.

Automation

  • Scripts - Python for data work: merging exports, reconciling figures, producing the monthly file that used to take a day; PowerShell for Microsoft 365 and Azure; version-controlled and scheduled.
  • Azure Functions, Logic Apps, Power Automate - Functions for code that only needs to run when something calls it: a notification, a transform, a small integration; Logic Apps and Power Automate when the workflow lives inside Microsoft 365 and business users should own it.
  • Every job reports - outcome, counts, timings to monitoring; failures alert someone the same day.

A subscription automation platform isn't automatically simpler than a short script somebody owns and can read.

How we work

Every project follows the same sequence:

  1. 01

    Discovery - which decisions the data serves, which sources exist, and what state each one is in.

  2. 02

    Design - we agree the platform, the schema, and the validation thresholds with you on paper; if something will not work, we say so before it is built, not after.

  3. 03

    Build - one module per source on a shared pattern; the schema documented as it is created. You talk directly to the engineers building your platform, with nobody in between, and we show you where the work stands against the schedule whenever you ask.

  4. 04

    Run - scheduled, monitored, alerting the same day; runs that fail validation never reach the warehouse.

  5. 05

    Hand over - the code, the schema, the dashboards, and notes on each source.

  6. 06

    Support - we stay on an SLA for as long as it is needed: until the day your own IT takes the system over, or for years afterwards. The length is yours to set - we tie nobody to a term they did not choose.

What lands in your hands

At handover you get:

  • The pipeline code, in a repository you own
  • The warehouse or lakehouse schema, documented
  • Dashboards built around your decisions
  • Run logs and alerting, wired to someone who will see them
  • Per-source notes: where each one comes from, and what breaks it
  • The validation thresholds, written down
See this in practice
ENPL