Guys I have a dataset with a time series regression problem but the target variable has 95% values missing.. I do know about SMOTE, KNN and dropping those values.
But should I even proceed with such data? 🥲
Can sone expert help me with this?

PS: already posted on Quora, stack, g4g but not much inputs

like
Posting as :
works at
You are currently posting as works at

Since you mention time series regression problem, curious to know what the missing values are.. by definition, time series is taken at definite interval(most of the cases), and if you have missing data there, it might mean that the observations were not recorded? What kind of data are you working with?

likehelpful

yes target column values were 95% missing, rest of the column values were all present..

How much data do you have in 5%.
Also is that all recent data?

no, that’s not recent even.. totally random 🤯

95% missing data , is it just for dependent variable ? If thats the case change dataset ? U can add categorical feature for the 5% and check if its working .. but ...

If I were you I would have did following things .
1) if it's time series data identify trends for available data and scrap the data from the internet ( if suitable data available ) and create custom dataset show it to to mgmt and ask them for updates , create a model and show them how the prediction can go wrongs ..
2) shows model scores or enjoy looking outside the 🪟

Are you doing AR model or VAR.

Timeseries data will have major implications on missing data... and you need atleast 4 years or 4 complete cycle of data for some proper analysis..
If u are doing smote or any other technique it might fill data.. but loose the essence of it.. i would say try to understand the business or ask an sme to fill the data for you.

Related Posts

Do we really have lay offs happening in zs ?

like

I’m taking BEC on Saturday. I have 4 more chapter to study. Do you think this is doable or should I move my exam date?

Hey I’m actually a security officer, although they don’t offer that option. And was wondering if anyone else here was one.

like

Hi,

Two weeks back i got a mail saying that i have been selected for legato. Till now i have not received any offer letter. Can anyone please suggest me what should i do or whom should I contact?

like

Who billed the most today?

like

Anyone move from a Big 4 to Big 3? How long were you at Big 4 before moving? And how’d you manage the transition?

Currently making ~$300K TC in a strategy role. Was just offered a different role (account manager) in the same company with a slight increase in base salary and a 50% TC increase if I hit sales target (even base case would be ~30% increase in TC).

Have never really considered account management / anything with a sales target, but I like money.

Thoughts? Things I should consider?

like

Hi guys,

I got my full and final settlement today but I can see no tax deduction. Does anyone know the reason. 4k tax used to be deducted every month till now. I also got gratuity and leave encashment along

like

I feel like I am getting more judgmental and a bit less compassionate as I get older. Can anyone relate, and any tips to help stop this way of thinking?

likehelpful

Hello Fishes,

What will be my in hand salary pm for the below salary structure. ?

Total CTC offered 13.5 lpa including variable pay.

Post Photo
like

Who else has sent the ♥️ or 😂 emoji to your colleague / boss by mistake, then panicked while changing back to 👍

Post Photo
likefunny

Hey fishes! Anyone working at Meta who’s open to giving me a referral?:) would really appreciate it!

like

Is having too much “attention to detail” a bad thing?

like

- 8 Years of experience
- Power BI | MySql | MSBI | SSIS | SSAS | AWS | DAX | Data Modelling

Any opportunity ? Openings ??

Is Vertcoin one to short given that miner’s fees are halved?

Hi, what is the market standard for DevOps Engineer with 4+ yoe. Current is 11lpa fixed and 2.31 as variable

like

Openings at Accenture for various levels

Post Photo
like

Healthcare exit opps for $200k+ remote or hybrid (in Dallas)? 8 YOE. MBA Candidate

likefunny

Do any of you ladies have in-laws who live close by & insert themselves into your lives ALL THE TIME? I feel so overwhelmed by them, but my husband thinks it’s NBD. (Continued)

Hello everyone.

This is a thank you post.

This platform has helped me in a way to get more than a 100% hike.

I am working as a Data Scientist with 5.6 yrs of experience in DS/ML.

My current CTC was 12.6 LPA and after switch, I got 32.2 LPA (30 fixed).

So I would like to thank everyone who posts comments and share their experiences. This helps a lot to understand the current market scenario and whether they are underpaid or not.

PS: Attaching my CTC

Post Photo
likeupliftinghelpfulsmart

Additional Posts in DataScientists Network

Hey there Data folks. Happy new year to all!
Wanted to know what would be average salary for 3 YOE for Data Analyst role at Gartner Mumbai??

And how is Gartner in terms of growth, culture, work.. your inputs are valuable

like

Hello fishes
Can anyone reffer for data scientist fresher.
Need it urgently.

like

Which is better company to join as senior consultant for analytics and data science, CAPCO or Fractal?

like
like

Is there anyone here who has taken CloudyML course, if yes kindly share how is the course.
Is it worth the pay?
I am looking for practice and hands-on to prepare for interviews.
Thanks In advance.

like

Anyone worked on assortment Optimization on the basis of attributes and not on customer in E-commerce can you help which algorithm to use for this use case

like

Please suggest platform to practice SQL, which can build confidence for interview..... TIA

likesmart

Hello folks,
During switching, is the ctc given based on relevant exp only OR total exp and yoe ? i.e. i have only 2+ yrs exp as data scientist (internal switch) and rest is in automation. so will the ctc be given considering me as with only 2+yoe as data scientist or will they consider my prev experience also? cctc 9lpa total yoe 8yrs

like

Anyone appeared or cracked the @Morgan Stanley data scientist interview, recently?

like

Which data science course would you suggest?
1. Scaler Data Science
2. IIITB course and John Moore University
3. BITS Pilani DS course
4. Leave these course of PGD and go for MS altogether.

Currently having an experience of 2 years and 4 months with experience in Data Science, currently working as a Business Analyst at Axis.

Thinking of studying side by side and upscaling if possible.

like

Hello fishes,

I'm a data engineer working on our in house ETL tools and python scraping and I don't have any cloud knowledge. Is it good to learn "Azure synapse analytics" to upgrade and get into bigger roles with better packages. My yoe is 1 in data engineer role along with 4 years as content Analyst.

like

I've 10+ yrs of experience in Web Analytics, Site Personalization, SEO and CMS. I'm very much interested to move into Data Science domain. For a beginner in DS what are important skill set i should get to transit into DS? Thanks!

like
like

Hello Fishers,

I am having 12 Years of Experience in Technical and Operational Management & Recently I have done ML/DS and Done my PMP Certification.

Please suggest to me how to switch my job and how much I can expect Package. At present my CTC is 14 Lac PA.

I Need Suggestion and Support. Please Help me how to change my Path and how to put it in a track.

like

I have 1.5yrs experience as a data analyst and looking for a switch, current CTC is 8.0 LPA. What salary can I expect at this switch, is there any% u should look for?

like

11 likes for dm please

like

Hi fishes,

Can you provide some good online (preferbly free) sources that have data science/machine learning unstructured interview problems??
Want to prepare for a ds interview

like

Accenture or McKinsey?
What's better for data science/cloud roles?

like

Hi guys actively looking for data science openings . Have experience of almost 2 years in IT industry . 2019 passout . please help and like this post that allow me to chat with others .
Thank you.
PS- I am new to this platform here.

like

Can anyone please refer me for a data science job?

like

New to Fishbowl?

Download the Fishbowl app to
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
  • Discover what it’s like to work at companies from real professionals
  • Get candid advice from people in your field in a safe space
  • Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.

Already a user?
Login here

Share

Embed this post

Copy and paste embed code on your site

Preview

Download the
Fishbowl app

See what’s happening in your industry
from the palm of your hand.

A phone with Fishbowl app

Send download link to your phone

OR

Scan your QR code to download
Fishbowl app on your mobile

By continuing you agree to Terms of Use and Privacy Policy.

Messaging rates may apply

Download app

Sign up for free to view this conversation on Fishbowl

By continuing you agree to Terms of Use and Privacy Policy

Already have an account? Log in

Sign up for free to continue using Fishbowl

By continuing you agree to Terms of Use(New) and Privacy Policy(New)
Messaging rates may apply

Already have an account? Log in

For account settings, visit Fishbowl on Desktop Browser or

General

Legal