I’m quite new to data engineering , have a possible dumb question about data warehousing. I learned a lot initially about schema design to the 3NF but I’ve been reading books on facts and dimensions. Question: is loading data initially into 3NF structure and then afterwards Denormalizing a bad idea?

like
Posting as :
works at
You are currently posting as works at

Ok first of all there’s lot going on there and it feels like you are complicating unnecessarily. Ideally denormalized to normalized is the approach but rev could happen if necessary, but I don’t see the reason to do so.

like

Everyone starts somewhere man, if data size really dosent matter I’m going to download 5-10gb and start learning from books. I don’t exactly have a great environment in my workplace to be getting any more information other than reading the code that’s already written. No tickets . Documentation

The second half of your penultimate comment totally put me into confusion as I could barely understand what you are trying to say. There’s no right or wrong approach. Just make sure whatever you are planning, keep your manager in loop on your ramping up approaches. Your supervisor knows the workflow much better as every company has their own tailor fit way of doing and they can guide you better if you ever were to run into pitfalls. If they don’t know any once of it then it’s time to take matter in your own hands and start writing up some onboarding docs and get a supervisor stamp on it. It will leave a long lasting impression and helps you grow up the ladder including $$$.

My 2c to you. I sympathize your situation if there’s no mentor in your team because if anything goes wrong then saying “Im sorry” will be too late. Be careful and test your content prior to deployment.

A few suggestions which I'm highly biased here.

1. You can find a high level DE roadmap her e- https://github.com/datastacktv/data-engineer-roadmap. Now don't get overwhelmed by the sheer size of tech stack but be sure to master at least the bare min. concepts like Data Structures and Algorithms, Java/Python/(or any OOPs), SQL, Performance Tuning, Pipeline Building, Indexing, Sharding, Partitioning, Bucketing, etc.

2. Here's a good post I suggest to noobs (below). You don't have to pay for videos but there're already enough MOOCs available for mastering domain concepts. They also suggest how to get your hands deep into code.
a. https://medium.com/coriers/data-engineering-roadmap-for-2021-eac7898f0641
b. https://understandingdata.com/the-data-engineers-roadmap/

3. Take advantage of your workplace Saas, Paas, or Iaas to deal with real life business problems and if needed get approval from your supervisor.

4. Unless you aren't the only DE, see if you can talk to your peers and understand contributed to the team.

Good luck

Related Posts

like

Hi everyone. Interested in getting referral in Microsoft. Please let me know if you’re willing to help and I can share my job role and + CV. Can I get likes to unlock DMs as well?

like

What’s the typical pay gap between federal and private practice?

like

Hi all,

What would be take home for 22LPA in tcs? Post taxes and pf.

Thanks

like

Posting here since I know I’ll get shamed further in other bowls and I’m already feeling shame. I have noticed that panic turns into shame spirals for big projects, especially when someone is counting on me. This stems from letting other people down in previous jobs or in academic spaces, turning work in late, or turning work in that’s poor quality.

It ends up being a self fulfilling prophecy. I was told I have a month or so to complete a project, and now I’m week 6 on it and not done. Advice?

like

Hi everyone, I am interviewing for a position in the compensation practice which is under TaxAnd, and was wondering if anyone has any insights into how the work is and what comp is like. Thanks in advance!

like

What's the max fixed CTC Accenture can offer to career level 9 Team Lead for a hot skill ?

Please confirm. My current CTC is 26 fixed.
What max fixed CTC can Accenture give ?

Accenture

like

Can someone refer me to Experience TAS role at BDO USA, LLP ? I live in TX. I’m only seeing positions for west region and southeast.

like

How is the work culture in Deloitte Touche Tohmatsu India LLP? Is traveling mandatory in Deloitte Touche Tohmatsu India LLP. Who are the clients and can I work from home?

like

Anyone in here from the boogaloo?

How nice and affordable is the Downtown Crossing, Financial District and Chinatown area?

Starting at McK soon. Just heard you can't bring your own device to the cell phone plan. Also heard it's an outdated iPhone. What does this mean for Android users? Does this also mean I'll be lugging 2 phones around?

like

Anyone here is really good with financial modeling or in the oil and gas industry?

I am thinking about a clay face project for January 3rd - 6th grades. Any ideas?

Looking for flatmates in Gurgaon. If any associate is looking for flat/flatmates do hit me up

like

Hi, I am looking for job opportunity. Work ex-6. 6 years, tools skilled in OBIEE, OAC/OAS, NP: 30 days. Please feel free to dm me Or comment for any lead.

like

Is there any Muslim organizations at Deloitte?

like

Samurai is my big boy! He's so handsome but super skittish. Obsessed with my husband and will just sit in front of him and stare lovingly. 😍 He's our 2nd polydactyl rescue and his murder mittens are HUGE!

Post Photo
likeuplifting

Hi everyone! Can someone tell me the best companies that allow employees to work outside the country with might restrictions? I work at major FAANG tech company currently and I want to see my options. Thank you!!

Additional Posts in Software Engineering

How do you advocate for a "level" promotion in a company that has vague unofficial descriptions of levels? I'm doing L4-5 (on standard levels, or L3 within our system) work but can't get away from the unofficial "L2".

Making 140k with 7 yrs experience in this role, and 12 prior years experience in Systems Engineering roles. I know someone on my team is at 180k with a similar career.

Fighting with my manager atm on definitions of levels and feel like its a red herring.

like

Serious question from a happy software engineer, what do you all love about your work/actually enjoy doing during working hours?

I'll start; I'm privileged enough to have been given the opportunity to work and learn deeply on Linux and Kubernetes, and I get to tinker and play with them all day. Problem solving with them is a "fun" part of my job.

Of course most things aren't roses and sunshine, but I'd love to hear the positive parts of what you are doing, or what you like to do at work.

like

If you’re a SQL expert, plz plz comment so I can message you about a query I need help with. Thank you!

like

I got an offer from OpenText Canada for a Sernior Java Developer. How much is the salary for such positions? Is 70k per annum low?

like

Are there companies that would pay for a masters degree for a current employee?

like

I was wondering if any technical folks had ideas for an Augmented Reality App or project, I’m trying to make something for my portfolio for a job but can’t come up with any ideas. I do ML at Google so I could incorporate image, music, text recognition into it, any ideas?

like

What makes you want to learn yet another programming framework / library / language besides the stack at work?

like

Have been struggling to context switch between different projects that I am working on simultaneously. Any tips? Thanks in advance :)

like

I’m a mid level engineer and my team is overwhelmed. There’s more work than we can complete in the timelines that leadership/product has committed to, which is causing leadership to override technical decisions seemingly to meet those deadlines. An example would be routing all live traffic to a feature which was not fully tested, instead of waiting until testing is complete or having that change be dormant and the feature enabled later. Has anyone dealt with this type of issue?

like

Hi all! What are the differences between senior/staff SWE and software architects? Are architects still a thing? I'm causally searching for a new position (my job had a 50% layoff last month and the survivability of the company is sketchy) and I've noticed senior and staff SWE roles sound like they have a lot of architectural designing and leadership responsibilities while not focusing on coding as much. Is this normal? I love mentoring, but I also don't wanna be in meetings all day. 😕

like

Hi, I changed my job like 3 months ago, it was supposed to be a lead front end engineer position, but after and awful onboarding I’m just a regular dev in a project that is not yet live but requires a lot of refactor. It was supposed to be a project to get familiar with the company but at this point I don’t know what to do. Give them some time, talk with them about my situation or just look for another job. What do you think about it?

like

Starting a role in software as a Product Owner. Should I have a Mac or a PC?

like

For people who love reading technical books on software architecture, design, programming languages and frameworks, etc - what do you read your books on, paper or screen? I find ebooks convenient for most of the things as it is easier to search for notes and annotations. On the other hand, paper books are a great companion by the desk for a quick reference.

like

I'm thinking it's time to start looking elsewhere. I've got 9 YOE mostly Java and Node backend with Angular frontend. It needs to be remote. I would love MAANG, but the idea of interview prep for big tech sounds terrible. Where should I look?

like
like

Has anyone done the JPMorgan Chase Emerging Talent Software Engineer Program? What did you learn in the program? What is the salary? Is the salary the same for the entire program; 2years?

like

CS and SWE is great in that it can be applicable to a great range of fields, to create immensely helpful tools for other things.

What are some areas where SWE intersects with things that help people, country, and the world? Now also add in making a decent living and generally competitive with other SWE “industry”.

Things that come to my mind are government-related, military, some fintech. Sciences have great need of computing, and can have big impact, but income is less clear.

like

Hello everyone! I need some guidance if you can share some. I do presales and I'm excited how most tools are finally embracing APIs. I don't have a dev background. That means I am becoming obsolete.
What do you recommend to close the knowledge gap?
Straight learn dev?
Learn JavaScript focusing in API?
Learn TypeScript focusing in API?
Lean Python scripting than API?

like

Can you practice for leetcode without knowing how to code?

like

New to Fishbowl?

Download the Fishbowl app to
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
  • Discover what it’s like to work at companies from real professionals
  • Get candid advice from people in your field in a safe space
  • Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.

Already a user?
Login here

Share

Embed this post

Copy and paste embed code on your site

Preview

Download the
Fishbowl app

See what’s happening in your industry
from the palm of your hand.

A phone with Fishbowl app

Send download link to your phone

OR

Scan your QR code to download
Fishbowl app on your mobile

By continuing you agree to Terms of Use and Privacy Policy.

Messaging rates may apply

Download app

Sign up for free to view this conversation on Fishbowl

By continuing you agree to Terms of Use and Privacy Policy

Already have an account? Log in

Sign up for free to continue using Fishbowl

By continuing you agree to Terms of Use(New) and Privacy Policy(New)
Messaging rates may apply

Already have an account? Log in

For account settings, visit Fishbowl on Desktop Browser or

General

Legal