Cloud Based Data Lake - Database Management as a Service

60%

Status

Executing [Implementation of the project]

60% complete, updated on Thu 9/7/23 2:39 PM by Lance Rivera

Changed Percent Complete from 55% to 60%.
 Compunell Project Tracking

COMPLETED

  • Conducted workshop on Microsoft Azure on 08/29, (this will help TSTC to set up Azure ecosystem)

IN-PROGRESS

  • Gathering information around missing column to column mapping for financial cube.

  • Meetings with Finance SMEs to understand their usage, challenges, vision etc.

  • Requirement gathering for the CFO Dashboard.

  • Engaged in preparing preliminary tasks for Azure Pipeline design POC.

  • Financial Cube development.

  • Finance Cube Requirement Specification Document and User Manual.

  • Data model analysis for Cube and data visualization.

TO-DO

  • identify all the possible domain areas where we need to design the cubes.

  • Discussions related to NCM and Tuition forecast with TSTC Finance team

  • TSTC to start raising production support tickets

Details

Dates
Mon 7/18/22 - Tue 4/30/24
Acct/Dept
Office of Information Technology
Type
Solutions Project Management / Default
Health
Green - On track
Created
Mon 7/18/22 10:29 AM
Modified
Thu 9/7/23 2:39 PM

Project Request Form

Requestor
This is an individual that is listed as the person asking for a project evaluation, may not necessarily be the Project Champion.
George Makiya
Project Champion Supervisor
Project Champion's direct Supervisor
Jonathan Hoekstra
Vice Chancellor
Project Champion's Vice Chancellor for the division
Jonathan Hoekstra
Requested Delivery Date
Enter an estimated date you would like this project completed/delivered
11/30/2022
Do you have an existing solution you are wanting to replace or enhance?
If so, provide some background information on the solution / process you are currently using.
No
What are your goals and objectives this solution will accomplish for your department?
Please provide your goals and objectives this solution will accomplish for your department.
The college desires an architectural review of our existing data and reporting environment, as well as skilled resources to rapidly configure and develop the analytic models needed to realize the value of the BI technology investments.
List your primary requirements for this solution.
Please list your primary requirements for this solution.
Prospective vendors should clearly demonstrate their capabilities for provision of the aforementioned services. The following are the minimum requirements.
They should demonstrate experience in conducting architectural design and performance assessment of the Microsoft Business Intelligence stack. The College runs on SQL Server 2019 and has licensing for the entire stack. However, only the ETL tool, the database and the Warehouse are in use. The vendors should demonstrate expertise in designing an end-to-end solution leveraging the college’s existing investments.

SSAS multidimensional development is purely for the on-premises datasets for reporting and analysis purposes. The scope does not include SSAS tabular models, which will be implemented on the cloud platform. A key requirement will be experience and/or certification in Microsoft Multidimensional Expressions (MDX) language that is used in SSAS.

As the Data Lake is implemented, some data that already exists on other cloud platforms will require to be Extracted, Loaded and Transformed (ELT). This requirement is critical for the smooth integration of disparate data types. The architectural design should also address acquisition of unstructured data into the Data Lake. For the time being, the college favors Adobe Spark, though open to suggestions/alternatives
Do you have funding for this project? If so, please provide the amount.
Please list Budgeted Amount?
Yes - Guaranteed funding
Please list any subject matter experts who can assist with this project.
Who has the knowledge in your area or other areas that can assist?
George Makiya

Description

TSTC is, as part of its digital transformation efforts, migrating its data and reporting environment to a cloud-based Data Lake. The legacy systems of record such as Colleague as well as several others will however remain on premise. As part of this cloud transition, some of the on-premises database infrastructure tools and services will be upgraded and/or stabilized prior to full migration. The current database (DB) and Business Data Intelligence tools portfolio remains largely inefficient and/or poorly configured.

Tools such as the Extract, Transform and Load (ETL) require reconfiguration and fine tuning. The SQL Server database and warehouse require higher skilled expertise to tackle the chronic problems of data hygiene, quality, and traceability/lineage. The college has invested in data cleansing and management tools such as Microsoft Data Quality Service (DQS) and Master Data Service (DMS) which remain unutilized due to lack of skilled resources.

The reporting landscape is suboptimal and requires development of new analytical models using the SQL Server Analysis Service (SSAS). This project seeks to engage expert level resources to provide skills augmentation to review the architectural landscape, recommend improvements and/or reconfiguration, development of the SSAS models and provision of top-notch ongoing support for the ETL, and the Data Warehouse (DW).

Manager

Alternate Manager(s)

Stakeholders (8)

BH
Bill Holifield
Data Engineer
Responsible, Accountable
Senior Manager - BI Infrastructure
Fri 6/2/23 3:55 PM
GM
George Makiya
Executive Vice President Data & Analytics
Responsible, Accountable, Consulted, Informed
Exec VP, Data & Analytics
Mon 7/18/22 2:12 PM
GP
Gustavo Perez
Data Analyst
Responsible, Accountable
Data Warehouse Analyst I
Fri 6/2/23 3:56 PM
JH
Jonathan Hoekstra
Vice Chancellor/Cfo
Consulted, Informed
VC & Chief Finance Officer
Mon 7/18/22 2:11 PM
LT
Luan Tran
Applied Research Analyst
Responsible, Accountable
Applied Research Analyst
Fri 6/2/23 3:57 PM
MJ
Madelynne Johnston
Exec Asst to Chancellor
Consulted, Informed
Chief of Staff II- Finance
Mon 7/18/22 2:11 PM
NS
Nandita Singh
Director of Student Analytics
Responsible, Accountable, Consulted, Informed
Director - Student Analytics
Mon 7/31/23 9:52 AM
TS
Tina Skidmore
Senior Executive Director
Responsible, Accountable
Director - BI Center of Excellence
Fri 6/2/23 3:56 PM