Learning cloud architecture right now so apologies if these are obvious questions: Best ways to store video metadata to perform analysis on it? I'm looking at AWS and GCP. Saw that Cassandra would be good for it, but not sure what the GCP equivalent is? But also I'm a bit confused on if Cassandra is just a DB and storage would need to be elsewhere? All help is welcome! I've been spinning my wheels on Google trying to understand

like
Posting as :
works at
You are currently posting as works at

These are very valid questions. If you are trying to migrate database records as batch process, you generally would need to stage in cloud buckets and then update your database tables. These might differ across cloud vendor.

However, assuming once it’s setup, you should be able to query and update the database directly using an endpoint. There is no need to first load the data to storage and subsequent update to database. You could maintain replicas and cache as well as perform snapshots for faster reads and reliability.

This brings us to the question as how many hits one can make to the database per second and associated scalability constraints. For small and moderate scale workloads, sql are more than fine if you have a defined schema and query patterns do not include network like joins.

Not a database design expert but just my two cents on the topic.

likehelpful

Well metadata of media files. Media files would be stored in an S3 bucket, but trying to understand best way to store and analyze the metadata of those media files

like

Listing three to five words that describe your requirements may help you organize. Try picking from the below list, and then look up tools based on your priorities:
https://en.m.wikipedia.org/wiki/List_of_system_quality_attributes

like

Is the metadata schema enforced or could contain free fields?

Yes user info would be a separate DB - assuming something like AWS RDS or Google cloud SQL. I guess my confusion lies on whether that data needs to first be stored somewhere like an S3 bucket beforehand? And if this is ideally real time info, would it make sense to have things cached after/if being stored before querying info from a DB. Again apologies if I'm just totally off here, just started looking at cloud architecture and really trying to learn some different scenarios

Use cloud data store aka fire store to store metadata

Related Posts

Considering a role at BCG Digital Ventures - can anyone share the hierarchy and salary structures?

like

Google tse ( tech support + consultant)


Walmart data engineer

Target data engineer



Walmart Target Google Amazon

Which one do you all suggest?
Salary same for all

like

any great high school science ice breakers/activities to do on the first day?

Also, anyone have a great lab safety video they show to their class? Need a new one

like

Do you keep offering circulars/memoranda from a financial printer when you leave a firm? I have a few (some private, some public) and I’m not sure what to do with them.

like

If your firm is asking summers to make those cringey self-aggrandizing acceptance posts on LinkedIn. Can you kindly ask they stop immediately?

likefunny

Can I ask at what age people are typically buying their homes here in LA? Wondering at what age people are able to take the plunge.

like

Do you have advice for making industry connections as a fairly anti-social introvert?

like

Hi all,

I am joining Natwest Group in July 2022 and as per my agreement with HR and manager, I will be working from office after 2 months of joining (from September onwards)

My question is how will my onboarding be online or I have to be in office for onboarding process.

Your help is highly appreciated

like
like

How much would you give to this child 😏😉😊

Post Photo
likefunnysmart

Don’t make me go back to the office… 😩😩😩

Post Photo
likefunnysmart

I’m behind on my IG game & want to catch up. I went to a few events before COVID that I want to share but feel weird posting it now. Do I put throwbackthursday or beforeCOVID hashtags? Something else?

likefunny

I’ve been on Wellbutrin for about a year and it was working…but I feel the depression coming back. I don’t want to just keep upping the dosage (went from 150 -> 200). I’m not too sure what to do and therapy is something I can’t afford right now with my recent rent increase. Any thoughts or advice?

like

Creating this support thread to check how everyone is doing and feeling this week

like

35 / F / SF / 115k+

like

Newly appointed Sales director - over 6 years experience on the sales side - leading 5 other sellers. What do you think my salary should be - both base and commissions.

What’s the average $$ a mid level designer at an agency in NYC? I’m at 70k with 3.5 years experience. Is that to low or just right?

like

What is the present value of a government pension? How does one calculate it? Trying to get a sense of weather it's worth moving over to Federal.

like

Any CAA or tax expert here?

Additional Posts in Data & Analytics Consultants

like

To all my data engineering friends out there, do you all feel like trash sometimes? Like people dont understand shit about what you do, and clients just want to see some bs ml model/pretty but useless dashboards?

likefunny

Any cyber security folks here? or those interested in systems engineering? Potential role with TC 120-180, awesome team and wlb, at least compared to consulting.

like

What would you choose between palantir and a small startup with great growth opportunities?

like

Got Amazon career essentials (ACE) assessment to complete for Business Analyst role
To anyone who has experience with it- What kind of work simulation questions should I be expecting?
Is there a way I can prepare for it or any resources one must know of?

like

Can anyone at Ford Motor Company tell me more about the “Learning Analytics and Engagement Specialist” remote position? Very curious and would love a referral!

like

Have zero data experience. Best way to start self learning?

like

Meeting with a recruiter at one of the FAANG companies. I won’t be able to move jobs because of H1b /GC at least for the next year. Any idea if I should start off with that or should I just try to understand the work that they have been doing?

like

Any conda users in here have to install a binary and use conda forge? Need some help installing pdftotext. Throwing a weird message saying “Found conflicts!” Any and all help would be appreciated.

like

What tech companies are best for Data Analysts? Looking for $120k base + equity that’s gonna be worth a lot + other great benefits + possible remote.

funny

Anyone made the jump from EY DnA to Bain AAG (or vice versa)? I’m wondering how they stack up across technical capabilities, project mix, WLB, etc.

like

Of all the job pitches that I have received, this is the most incomprehensible so far.

Post Photo
like

Is anyone working with Elastic at DoD?

Hi fishes, can you guys share your favorite websites to look for remote job? To look for start up companies? TIA :)

like

How does everyone in this bowl feel towards master’s degrees in data science/analytics? I didn’t study this in undergrad so I’m looking to get an analytics degree but I’ve heard mixed reviews.

like

I feel like we’re changing tools on every engagement and I’m always burning through trainings trying to master one only to keep it swapped for something else. Any recommendations for books/trainings to help my understanding of the underlying methods of data science instead of just learning the tools?

like

Anyone here work or has interviewed for a data science role at DataRobot? Would love to get some insight. Third interview coming up.

like

Any recommended courses or books on customer care/cust success analytics or marketing analytics in general?

like

I'm hiring for an analytics manager role. M&A Healthcare focus. Effectively work with PE on "business diligence".

Revenue cycle management experience is a big plus. Not much ML used in the practice but I'm trying to change that.

In an ideal world you know ML, PowerBI, alteryx, and rev cycle. Bit of a unicorn ask.

If anyone is interested feel free to hit me up.

like

Noob question - I want to consolidate the dates into a single dates column. Would numpy or pandas help me reorganize this dataset effectively or should I do it within excel?

Post Photo
like

New to Fishbowl?

Download the Fishbowl app to
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
  • Discover what it’s like to work at companies from real professionals
  • Get candid advice from people in your field in a safe space
  • Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.

Already a user?
Login here

Share

Embed this post

Copy and paste embed code on your site

Preview

Download the
Fishbowl app

See what’s happening in your industry
from the palm of your hand.

A phone with Fishbowl app

Scan your QR code to download
Fishbowl app on your mobile

Download app

Sign up for free to view this conversation on Fishbowl

By continuing you agree to Terms of Use and Privacy Policy

Already have an account? Log in

Sign up for free to continue using Fishbowl

By continuing you agree to Terms of Use(New) and Privacy Policy(New)
Messaging rates may apply

Already have an account? Log in

For account settings, visit Fishbowl on Desktop Browser or

General

Legal