Call for a little help: working on a research project using PySpark for processing large quantities of data, and I am struggling to find a good setup that doesn't cause a stackoverflow error. Have checked all available resources but to no avail. Pls DM if you would be open to spend a few minutes on helping me out, eternal gratefulness from my side guaranteed!

like
Posting as :
works at
You are currently posting as works at

Where are you running spark?

like

As a python package in the Pycharm IDE, on a working station with 16GB of RAM. Was thinking to maybe try it on Azure with more storage/processing power but not sure whether that could help..

Don't share your firm's/client's codebase externally, big no-no on many levels. Talk to your team, find some other help inside the firm, or generalize your code enough to ask on StackOverflow

It is a publicly available dataset I am using for academic research, but thanks for pointing it out!And unfortunately noone in my team has any knowledge on those topics..

Can you show your error? Dm me

Use Polars instead

Related Posts

Trying to get my fat consumption up but don’t like the taste of coconut oil (most fat bombs are made of those) Any tips on what you snack on?

like

Girlfriend has very obvious high functioning anxiety and body image issues that she admits but thinks she doesn’t need therapy or medication and that reading self-help books will help her. It’s not changing in the last several years and it’s impacting our relationship. Anyone have experience with their SO dealing with untreated anxiety? I know I can’t control her but would like to express my feelings without sending her into a spiral. For context, I’ve been in therapy for 12 months and medicated

like
like

Bain & Company To consultants at MBB’s, Roland Berger, OW, Kearney, Arthur D.Little. When does the openings starts for Junior position.
As I’m checking out for vacancies, I can’t see any at the moment. Usually there’s a timing for when do they open. Any idea? Bain & Company McKinsey & Company Boston Consulting Group Kearney Roland Berger Oliver Wyman Arthur D. Little Limited

like

What are some 1-2 year experience roles at MAANG companies?

like

Hi Guys,
My joining date is on 26th December but I haven't received any mail regarding joining and orientation yet.
When can I expect the same? Any Idea..!!
Please help

like

Question: since August last year, the same recruiter at a mid-size agency has asked me about freelance gigs maybe 4 or 5 times......continued:

Has anyone had a client (agency) just not pay? If so, what steps did you take to get them to do it? They were already late before COVID and now they’re just not responding. Any recos?

like

We have some community group send in pizza, cookies, and coffee to our emergency room during COVID. Another doc literally started crying he was so grateful

like

Hi Everyone, my LWD is on 3rd Sept this friday. I need to submit my laptop. I am working from another city. Wanted to know if its safe to dispatch laptop via EY courier partner.

like

What other careers do you catch yourself daydreaming about? I, for one, would have loved acting as I loved theater throughout my life but alas fate had other plans. What's yours?

like

Alvarez & Marsal I saw a listing on LinkedIn that Alvarez & Marsal is looking for associates to join their diligence practice. Is there anyone who would be willing to talk to me more about their experiences in that practice? Thank you!

likehelpful

Welcome to the rage bowl 😈 go ahead and comment a situation that currently has you pissed.

likeuplifting

Tired of the superficial swiping apps? Good old fashioned text based personals ads Just like in the olden days. Keep it classy!

like

Got industry offer at bank, cyber. Would be going from 110 to 130. Perks are comparable. I feel like it is on lower end of comp based on Glassdoor and elsewhere. Considering countering at 135-140.

like

Did anyone file AOS and i140 together (assuming your priority date is current at the time of PERM approval)? What was the process like and how long did it take to get your GC from PERM approval date?

Georgia Tech Cybersecurity masters or the analytics masters? Currently in a cyber role at Deloitte. I was thinking it might be better to do the analytics master and get a CISSP. I feel like there is more value in the cissp than a MS cybersecurity

like

Anyone looking for Referral to Deloitte USI can DM me.

like

Getting an opportunity to move from ZS Associates India to IQVIA UK. Can’t compare the pay because they are quite far apart (ZS pays much more than IQVIA). Thoughts on moving from ZS to IQVIA in the long run?

like

Additional Posts in Data & Analytics Consultants

Big 4 Data Scientists, how did you exit to big tech firms? I’ve been unlucky with online applications and even referrals. I think it might be my resume or how I present myself. Can any free birds please give guidance?

like

Is 128k a fair base salary for Consultant in Deloitte Analytics & Cognitive. MS CS and about 3 yrs experience. In one of the fastest growing but midtier tech cities in the US.

like

When industries refer to business intelligence engineers, are they looking at a senior level or entry level? Some of these job descriptions seem rather outlandish in terms of years of experience required like 5+

like

today I choose violence

Post Photo
funnylikesmartuplifting

Anyone who has done masters in business analytics Singapore? Could you help on courses, Scholarships and careers and jobs. Looking for help.

Folks applying for Data and Analytics roles in the industry at M, SM, D Levels, what technical/functional topics do you prepare for on the Analytics side?

like

Meeting with a recruiter at one of the FAANG companies. I won’t be able to move jobs because of H1b /GC at least for the next year. Any idea if I should start off with that or should I just try to understand the work that they have been doing?

like
like

Single book recommendation for user research?

Goal is interview prep. Ideally focused on applications in tech.

like

Hi! Does anyone know what kind of knowledge is needed to crack the Data Scientist interview at Google?

I'm currently working in Marketing Analytics (primarily using SQL and a bit of pyspark). Apart from mastering SQL, I wanted to know from people working at Google what does it take to get there as a Data Scientist? (I'm very interested in Strategy Ops)

I plan to another 6-8 months in preparation and upskilling myself before I even apply.
Any help and guidance is appreciated! Thanks :)

like

New data offering: name.co, or namesoftware.com for the main domain? I’m leaning toward name.co, but I worry about not having the .com.

like

What are some interesting non-tech/insurance/advertising/consulting careers where data science is applied? Thinking of epidemiology and economics

like

How does everyone in this bowl feel towards master’s degrees in data science/analytics? I didn’t study this in undergrad so I’m looking to get an analytics degree but I’ve heard mixed reviews.

like

Business practice: "can you guys build a model that overlays where Coronavirus is going to break out with client sites"

likefunny

1) Hypothetically if you were looking to sabotage a colleague, can you code an excel document to throw off their numbers or calculations, (if you send them that document and they download it for use)?

like

Does anyone work at EY in their Decision Modeling team? Not asking for answers, but just wondering if someone is able to share what concepts I should be familiar with when prepping for the case study (i.e. do I need to know how to run an LBO or is a DCF sufficient)?



like
like

How should I move to tech as a ML Engineer? How did people here make the transition?

likehelpful

Any recommended courses or books on customer care/cust success analytics or marketing analytics in general?

like

How is Deloitte’s Analytics and Cognitive practice doing? Saw a couple of open Data Science roles open on their website, are they actively hiring

like

New to Fishbowl?

Download the Fishbowl app to
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
  • Discover what it’s like to work at companies from real professionals
  • Get candid advice from people in your field in a safe space
  • Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.

Already a user?
Login here

Share

Embed this post

Copy and paste embed code on your site

Preview

Download the
Fishbowl app

See what’s happening in your industry
from the palm of your hand.

A phone with Fishbowl app

Scan your QR code to download
Fishbowl app on your mobile

Download app

Sign up for free to view this conversation on Fishbowl

By continuing you agree to Terms of Use and Privacy Policy

Already have an account? Log in

Sign up for free to continue using Fishbowl

By continuing you agree to Terms of Use(New) and Privacy Policy(New)
Messaging rates may apply

Already have an account? Log in

For account settings, visit Fishbowl on Desktop Browser or

General

Legal