Is data engineering roles on the decline? Not hearing about the opening .Also with AI how does one keep up-to-date with the technology

like
Posting as :
works at
You are currently posting as works at

In 3 months NP I have attended 27(exact ) interview. So no decline

like

Yoe?

When everyone is becoming data enginer then this issue is bound to happen . I am from traditional datawarehouse era . Previously for DW we use to have team of size 8-10 people to maintain but with databricks automation u need just 2-3 people . Problem is that people are not realizing this and jumping the ship thinking that it is shortcut ticket to high paying jobs. Believe me there is no challenge in working with databricks and data engineering . Previosuly u will do performance tuning now databricks takes care internally , u will monitor failures and control it now DB does automatically . Wat you do is just write SQL and python commands and also maintenance part is gone so once development is done the team will be ramped down and also how many projects will be there to implement . Databricks has automated lot of parts . Ask some experience person then you will realise that you dont need so many people and still people from Java,dotnet , legacy tools ,QA testing everyone jumping the ship to become data engineer . Ask them are they enjoying it .Databricks came up with Delta live tables read about that then you will realise that more number of steps has been automated which people use to do manually . Automation has risk of job loss . I use to run team of 16 people now after migrating to Azure i just need 4 people thats it .Now wat happenes to 12 people .It has ripple effect . Wait 2 years you will lot of job loss in India . It is going to happen. They are cramming up Data engineering field and AI field too .

like

So what do you think will be the way forward.

But they’re not selecting or paying more unlike in 2022. They just gave hikes back then. Now everyone’s lowballing

True. Supply demand.

Nor decline. Just that interview are tough unlike covid times. They expect u knowing spark internals, how it works etc

Related Posts

Hi Sharks, Can anyone please share about interview experience/questions asked at PayPal Paypal for java developer 3yr experience?
Is it general core Java based or heavy DSA based? Please help.
Thanks

like

Deloitte US just announced layoffs. Any other B4 started layoffs too ?

like

Does Deloitte and PWC has a dedicated app sec pen-test team? What percent of the time do you travel? Do you guys work from home or need relocation to any place in US?

like

6’5 here, does anyone have any recommendations on a good desk that is adjustable to a standing desk? I find that a lot of standing desks have a listed max height below 6’5

any idea how is Somnetics ?

YYY, aren’t you on my level?

Post Photo
like

How are holiday party expenses handled at your firm? The MDs on my floor usually split the cost out of pocket.

like

Typical days on a difficult project with a difficult partner

like

PwC is getting a lot of hate out there :(

Looking for advice: would you stay in partner pipeline at Big 4 or move to one of the MBBs as a principal? I am hearing that with Big 4 getting into strategy more, MBBs will be obsolete soon thoughts?

likefunny

The President spent months railing against vote-by-mail and trying to sabotage the USPS in conjunction with DeJoy, and now you can see why

likesmart

Hey Accenture 🐠 🐟: How it working within the capability network? I am looking for opinions, reviews, and advice. TIA

like

I thought I really wanted to be a chemical engineer but the reality of the day to day has been disappointing to say the least… is this a common sentiment for a first job? Do I just have to keep going to get somewhere interesting?

like

Does anyone here have a Tesla? Should I do it???...

like

Diversification is the key. My portfolio includes a bit of most of these.

1. Debt/Liquid mutual funds (direct)
2. Gold
3. REIT (fractional ownership and distribution)
4. Equity mutual funds (ELSS, direct)
5. Direct equity (India)
6. US stocks and ETFs (tech & genomics)
7. Crypto currency (interest deposits)

helpful

Hello Fishes, I am working g in US based Insurance process company in Mohali from past 2 years and having total experience of 12 years in BPO/KPO. Currently my salary is 26.6 now I am getting offer from insurance based START up and they are offering me Assistant Team Lead and salary of 35k but they want immediate joining. Please advise.

like

Hello Fishes,

Got an email today to close Tata Mutual Funds as it's restricted entity. Basically I am from Advisory team and not from Audit or Tax department.

Do i really need to close the Tata Mutual fund?

like

Does anyone ever use their overtime meal to buy something else when at Walmart or Supertarget?

like

First time in LA for the weekend. Which all places do you recommend to visit in 2 days? (local attractions/restaurants/scenic drives etc.)

Additional Posts in Data Engineering

Hi,
I am working as Data Analyst. Now, I seen my interest towards Data Engineering.

Please help me to decide, Should I go with
Azure Data Engineer
Or
with Big Data Technology like Spark, Kafka, Hadoop, Sqoop, Airflow.

Suggest me some good course to start.

Thanks 😊

like

Hi Fish,

I am currently working in non tech as an analyst. I have learnt SQL AND Data warehousing concepts. I am thinking of switching to any data related role Data Analyst/Data engineer. Appreciate your views/thoughts on this.
Should I have to learn some more to switch or should take up any real time projects to start a career in the above mentioned fields.
Overall experience: 3y
CCTC: 4.2 LPA

THANK YOU

like
like

Hi All,

I'm looking for referral in data engineering.
I'm having 2 year of relevant experience in data engineering.

like

Hi All,
Can anyone give me some pointers on the interview process for data engineering role in PepsiCo?

like

What does col a /=/ ‘ONE’ :: “VARCHAR” do in Netezza. Join or checks for null or match to ONE ?

How can I use the same in Oracle database? Any suggestions

like

Hi, I want to learn how to build ETL pipelines with Pyspark and Azure. As right now i have experience in feature engineering only with SQL and Pyspark.

Can anyone suggest anything online from where i can learn and do hands on?@

Hi fishes,
Hope you are doing well. I am looking for a job having 3+ years of experience in Data Engineer domain/ Python Development. If your organization is recruiting for this position. Then let me know or forward my profile to your connections.
YOE - 3+ yrs
NP - 30 days
Skills - Python, SQL, AWS, Pyspark, Jenkins
Thanks a lot in advance!

like

Why Data Modeler role is on high demand these days?

Hello
Can I pl get help in SQL
Thank you

like

Hi fishes,I have around 5.2 yrs of hands on exp in data engg. and able to clear service based companies interviews easily.Now I want to switch to product based companies as I am looking for some good work,good salary but I am not really strong in dsa as it's not part of my daily work.I don't really use trees and graphs as part of my daily work.
How shall I switch to product based companies considering DSA is their 1st round itself.what level of DSA can I expect from companies for data engineer position?

like

Can you please tell companies name which uses azure for their data engineering projects

Hi all,
Can someone guide me on how should a person start learning for transition into a data engineering role at a beginner level. Currently I have known Python and SQL in my job. Not able to figure how to go forward?
Any advice would be great!!

like

What do you think are the new things to learn in data engineering in Azure platform

Hi people,Does anyone have given visa hackerrank interview recently? May I know the kind of questions being asked and their level?

Hi everyone ,need your guidance.

I am working as SAP BODS developer which is a GUI based ETL tool , at EY GDS having 1.3 years experience.
I wanted to shift to data engineering based on cloud technology, so wished to get guidance on what things to learn and is it possible to get into this role in other companies.

TIA .

like

I have selected as senior data engineer in Fractal analytics, Verizon.

Which to choose based on work life balance and growth and job security ?

like

What are some good product based companies to work as Data Engineer?
Good in terms of learning, interesting work and good work culture.

like

Hi guys I work in One of GCP partnered org. My role sr SQL developer I resigned the current job GOT ROLE in FICO child org as a solution integration consultant role -(AWS,PENTAHO ,SQL we will be using ) -offered CTC 25 fixed + 1L JB + 2.5 BONUS + 0.5 L WFH SETUP = total 29LPA if everything will go smoothly ( bonus vary with comp growth it can be between 1.8L to 2.5l .so final CTC will get down to 28LPA as well - Bangalore but WFH ).Current org retain with 25L +2 LP VARIABLE HYDERABAD COMPANY

like

New to Fishbowl?

Download the Fishbowl app to
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
  • Discover what it’s like to work at companies from real professionals
  • Get candid advice from people in your field in a safe space
  • Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.

Already a user?
Login here

Share

Embed this post

Copy and paste embed code on your site

Preview

Download the
Fishbowl app

See what’s happening in your industry
from the palm of your hand.

A phone with Fishbowl app

Scan your QR code to download
Fishbowl app on your mobile

Download app

Sign up for free to view this conversation on Fishbowl

By continuing you agree to Terms of Use and Privacy Policy

Already have an account? Log in

Sign up for free to continue using Fishbowl

By continuing you agree to Terms of Use(New) and Privacy Policy(New)
Messaging rates may apply

Already have an account? Log in

For account settings, visit Fishbowl on Desktop Browser or

General

Legal