Related Posts
Rajma is bae 😍

Naked running vs running naked

Additional Posts in Data & Analytics Consultants
New to Fishbowl?
Download the Fishbowl app to
unlock all discussions on Fishbowl.
unlock all discussions on Fishbowl.
Rajma is bae 😍

Naked running vs running naked

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Download the Fishbowl app to unlock all discussions on Fishbowl.
Copy and paste embed code on your site

Scan your QR code to download
Fishbowl app on your mobile

With data that size I would suggest looking into Qlik
Or I guess the first question is... Are you loading those excel files into a QVD and then utilizing that? A QVD file is typically 10-100 times faster than loading from source
Millions of records are nothing, it will depend how you are designing and of course you need to have enough memory to accommodate that much data.
Tableau, PBI and other systems like these should all do fine with that amount of data as long as your design is adequate. If possible, try and connect directly to the billing database itself rather than CSV exports - it will make refreshing the data much more straightforward.
Power BI can handle tens and even hundreds of millions of records. Even terabyte-scale workloads if you do the solution correctly. The question is more about the complexity of your calculations over the dataset and how much related data you're combining. You can also offload file processing to dataflows.