Related Posts
How to ace first impression during coffee
Additional Posts in Tech
Rising Star
Does anybody have experience working at Onfido?
Pro
New to Fishbowl?
Download the Fishbowl app to
unlock all discussions on Fishbowl.
unlock all discussions on Fishbowl.
We have Sales/Marketing/Customer Engagement team that are in continuous need of the data and BI team builts queries as per the requirements and for respective timeframe and provide them with the raw data. Additionally there are 1000s of reports that have been built and those are delivered via emails to different stake holders.
Now, what automation ideas can I suggest so that we could come up solution that utilises a tool or something that can help us in automating this SQL query preparation task. Since the data is huge and a lot of time of team members goes into this.
Even I am relatively new here, so I still don't have the complete idea of the process but if you folks have any prior experience of such scenerios please do help.
All the ideas are welcome.
Please let me know if some more clarification is required on anything.
I would look at building some sort of flexible, self serve system to replace the 1000s of reports. Automate the selection of fields, the filters or conditions and the result set. The users will eventually prefer this over the pre-built reports. Also, provide a method for their selections to be saved, recalled, copied and modified and they’ll love you.
... I can consult for food ...
Here are some automation ideas to streamline SQL query preparation and data delivery:
1. Self-Service BI Tools
Implement a tool like Power BI, Tableau, or Looker to allow users to generate their own reports with predefined data models.
This would reduce the BI team's effort in writing custom SQL queries.
2. Parameterized Query Templates
Create dynamic, parameterized SQL queries stored in a query repository.
Users can input required parameters via a web UI or a chatbot to generate reports instantly.
3. Automated Report Scheduling & Delivery
Use SQL Server Reporting Services (SSRS), Power BI Report Server, or Tableau Server to schedule and deliver reports automatically.
Reports can be sent via email or accessed through a dashboard portal instead of ad hoc requests.
4. AI-Based Query Generation
Implement natural language query (NLQ) tools like ChatGPT API, Google Looker Studio, or Microsoft Copilot that allow non-technical users to describe their data needs in plain English.
The tool translates this into an optimized SQL query.
5. ETL & Data Warehouse Optimization
Create a centralized data warehouse where commonly used datasets are pre-aggregated.
Use ETL tools like Apache Airflow, DBT, or SSIS to automate data extraction and transformation, reducing the need for manual SQL query execution.
6. APIs for On-Demand Data Fetching
Build a REST API that allows teams to request data dynamically instead of waiting for manual query execution.
The API can fetch data from pre-optimized views in the database.
7. Chatbot or Slack Integration
A chatbot integrated with Slack or Microsoft Teams that allows users to request reports in natural language.
The chatbot can use AI to generate SQL queries and fetch data from the database.
Have you considered Looker?
It’s free basically
They could use SSIS packages to do this or they can schedule jobs using the agent and also use powershell to complete this. They are already using SSIS correct?