Learning cloud architecture right now so apologies if these are obvious questions: Best ways to store video metadata to perform analysis on it? I'm looking at AWS and GCP. Saw that Cassandra would be good for it, but not sure what the GCP equivalent is? But also I'm a bit confused on if Cassandra is just a DB and storage would need to be elsewhere? All help is welcome! I've been spinning my wheels on Google trying to understand

like
Posting as :
works at
You are currently posting as works at

These are very valid questions. If you are trying to migrate database records as batch process, you generally would need to stage in cloud buckets and then update your database tables. These might differ across cloud vendor.

However, assuming once it’s setup, you should be able to query and update the database directly using an endpoint. There is no need to first load the data to storage and subsequent update to database. You could maintain replicas and cache as well as perform snapshots for faster reads and reliability.

This brings us to the question as how many hits one can make to the database per second and associated scalability constraints. For small and moderate scale workloads, sql are more than fine if you have a defined schema and query patterns do not include network like joins.

Not a database design expert but just my two cents on the topic.

likehelpful

Well metadata of media files. Media files would be stored in an S3 bucket, but trying to understand best way to store and analyze the metadata of those media files

like

Listing three to five words that describe your requirements may help you organize. Try picking from the below list, and then look up tools based on your priorities:
https://en.m.wikipedia.org/wiki/List_of_system_quality_attributes

like

Is the metadata schema enforced or could contain free fields?

Yes user info would be a separate DB - assuming something like AWS RDS or Google cloud SQL. I guess my confusion lies on whether that data needs to first be stored somewhere like an S3 bucket beforehand? And if this is ideally real time info, would it make sense to have things cached after/if being stored before querying info from a DB. Again apologies if I'm just totally off here, just started looking at cloud architecture and really trying to learn some different scenarios

Use cloud data store aka fire store to store metadata

Related Posts

I give the clients gold, but they decide instead, they want me to paint it with sh*t.

likefunny

So, some of my co-workers simply said that they want to go ahead and start something or start looking elsewhere (career change) because they don't know how much it will last at the pace we are going in regards to case intake and pay out. Should I start panicking or simply start looking elsewhere? They even they even told me if I was looking elsewhere or planned on doing so also. I found it weird in a way.

like

What’s the tea on TDH Chicago?

Starting in 10 days with D. When will i get orientation details and welcome emails? Do my manager reach out before I join?

like

Hi fishes
I put down my papers yesterday. Now i am anxious. If anyone can please help me in getting a profile in Management consulting or operations. I am done with sales roles. I will b grateful.
TIA

like

Hi fishes,
I'm an engineering grad (2021) from Mumbai University, placed in EY thru campus.
I'm looking for referrals in Business Consulting for freshers.
Please comment so I can DM you.

Thanks in advance :)

like

what’s the best way to ask for a raise during your annual review?

like

Any recommendations (or watch outs) for au pair agencies? CT/NY area if it matters. TIA!

like

What is the TC for a partner at PWC?

like

How many hours would you work when you have your 2 weeks in? Would you work weekends?

like

Hi folks, recently got an offer from birlasoft. Wanted to know how’s work culture and what clients available in chennai location?? Any feedback would really help, thank you. Pls advise

Anyone willing to refer me for a Business Analyst position @ Amazon?

Anyone who moved from offshore audit practices to industry for better WLB?

like

My wife often calls me by 5:45/6 to ask where I am if I’m not headed home yet. Sometimes she’s stressed with kids/dogs/dinner and really wants me home. This really throws off my concentration because it makes me feel rushed. Like many practices sometimes I’m out at 5:30, other times it’s much later. Just depends and is unpredictable. Any advice on how to approach this?

like

Hi everyone. I saw on insta that some ambassador elite got gifts. I only got the voucher despite getting ambassador this year regardless of covid extension. Am I going to get something?

like

Is it SAFE joining Wipro as technical lead (B3) with 37LPA? Knowing that package is pretty high as compared to collogues in same band. HR is not ready to give project manager(C1) position.

Appreciate your suggestions/input.

Would there be any scope of increment going forward?

YOE 8.7

Tech SF DevOps

Wipro Infosys HCL Technologies Tata Consultancy Cognizant

like

What is salary range(break up would be helpful) for Software Engineer 2 position at Microsoft India ?

What would be the minimum monthly expenditure for a single person in bangalore considering rent, food and other essential expenditures ?

Additional Posts in Data & Analytics Consultants

Book recommendations for Experimentation (for a noob)

like

What tech companies are best for Data Analysts? Looking for $120k base + equity that’s gonna be worth a lot + other great benefits + possible remote.

funny

Of all the job pitches that I have received, this is the most incomprehensible so far.

Post Photo
like

Hi fishes, can you guys share your favorite websites to look for remote job? To look for start up companies? TIA :)

like

What is the best way to handle group financial consolidation in a multinational company?

like

Noob question - I want to consolidate the dates into a single dates column. Would numpy or pandas help me reorganize this dataset effectively or should I do it within excel?

Post Photo
like

I feel like we’re changing tools on every engagement and I’m always burning through trainings trying to master one only to keep it swapped for something else. Any recommendations for books/trainings to help my understanding of the underlying methods of data science instead of just learning the tools?

like

Got Amazon career essentials (ACE) assessment to complete for Business Analyst role
To anyone who has experience with it- What kind of work simulation questions should I be expecting?
Is there a way I can prepare for it or any resources one must know of?

like

Any cyber security folks here? or those interested in systems engineering? Potential role with TC 120-180, awesome team and wlb, at least compared to consulting.

like

What would you choose between palantir and a small startup with great growth opportunities?

like

Any recommended courses or books on customer care/cust success analytics or marketing analytics in general?

like

Single book recommendation for user research?

Goal is interview prep. Ideally focused on applications in tech.

like

To all my data engineering friends out there, do you all feel like trash sometimes? Like people dont understand shit about what you do, and clients just want to see some bs ml model/pretty but useless dashboards?

likefunny

What are the top sales & marketing metrics that tech companies track? I have an interview for a Data architect role and the data aspect is my strength but need to brush up on the business insight part

like

from God import pandemic
From China import coronavirus, mortality_rate

covid_19 = pandemic.coronavirus.set_mortality(mortality = 1.)

like

Any conda users in here have to install a binary and use conda forge? Need some help installing pdftotext. Throwing a weird message saying “Found conflicts!” Any and all help would be appreciated.

like

What's the best way to compare the similarity of text between a smaller set of documents with a larger set of documents? Therefore, two vectors of vastly different sizes.

like

Can anyone at Ford Motor Company tell me more about the “Learning Analytics and Engagement Specialist” remote position? Very curious and would love a referral!

like

Has anyone else begun to resent data science?

like

Meeting with a recruiter at one of the FAANG companies. I won’t be able to move jobs because of H1b /GC at least for the next year. Any idea if I should start off with that or should I just try to understand the work that they have been doing?

like

New to Fishbowl?

Download the Fishbowl app to
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
  • Discover what it’s like to work at companies from real professionals
  • Get candid advice from people in your field in a safe space
  • Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.

Already a user?
Login here

Share

Embed this post

Copy and paste embed code on your site

Preview

Download the
Fishbowl app

See what’s happening in your industry
from the palm of your hand.

A phone with Fishbowl app

Scan your QR code to download
Fishbowl app on your mobile

Download app

Sign up for free to view this conversation on Fishbowl

By continuing you agree to Terms of Use and Privacy Policy

Already have an account? Log in

Sign up for free to continue using Fishbowl

By continuing you agree to Terms of Use(New) and Privacy Policy(New)
Messaging rates may apply

Already have an account? Log in

For account settings, visit Fishbowl on Desktop Browser or

General

Legal