Four months into the project, the architecture was completed and the intent was fairly well defined but data ingestion was overly complex. There were leaking elements when reconciling Curated and Reporting layers with RAW, and inconclusive root cause. The previous consultancy had adequately built a foundation for the data lake leveraging Azure and Databricks, but there were recurring complications ingesting and mapping data, and consequently, their contract was severed. Additionally, the internal supporting data engineers were new to agile principles and shared resources across other projects outside of this initiative. The team did not have a product owner but had a PMP-certified project manager playing the role of Scrum Master.

Ippon was contracted for project rehabilitation. We would have until the end of the year, or just under four months, to audit the current situation, possibly design and execute a new destination state, and support successful data ingestion of 7 key sources, while training the surrounding team and improving their use of agile methodology.

This model was an embedded coaching, hybrid team augmentation. It was also somewhat unique in comparison to many Ippon enterprise deployments. Often we may add a full stack ‘Team-in-a-box’ to accelerate work while demonstrating and upskilling other full teams on the same initiative.

This client team had limited experience with agile and included 1 Project Manager (the org was traditionally waterfall) playing Scrum Master, 3 Data Engineers, and 1 senior infrastructure Engineer, all cross-functional shared resources on other projects. The team also included 3 dedicated QA analysts working off-shore, 10.5 hours ahead.

Ippon embedded a Sr Data Tech Lead and a Sr Product Coach, both at 50% time, 2 data engineers full time, and 1 cloud engineer to support CI/CD for a total of 6 weeks. We augmented to create a single, blended acceleration team.

Ippon ultimately succeeded in launching the data lake MVP to support its first use case reports, including automating the first seven required ingested sources aligned with all security requirements. Two engineers were retained for an additional month to triage and resolve data mapping errors.

More than a successful product launch, however, we may celebrate the organization's enablement more. Through coaching, the team developed strength in story creation, pointing, and execution. In the final sprints, team velocity eclipsed 85%, reflecting the team’s growth in efficiency. Overall time in meetings had reduced, and forums were more focused on value. The project manager was operating effectively as a Product Owner, building the backlog around the next Use Case intent.

Top 10 private equity firm uses a data lake transformation to also pilot agile development. Their first attempt loses steam, but Ippon takes a different approach.