CST 383 - Intro to Data Science | Week 2

 Learning log 2:

This week I learned more about pandas and how it is used to work with data in a more organized way. Last week we used NumPy arrays, but this week pandas Series and DataFrames made the data feel easier to understand because the rows and columns can have labels. I learned that a Series is like one column of data with an index, while a DataFrame is like a full table with rows and columns.

One topic that made more sense after the labs was indexing. With a pandas Series, I can use dictionary style indexing like mpg['Ana'], or I can use .loc to get values by label. With DataFrames, I practiced getting columns, rows, and specific values. I also learned that pandas lines up data by index, not just by position. That was important in the series lab because one student was missing from the distance data, so pandas returned NaN for that calculation.

I also learned about aggregation, which seems like one of the most important skills so far. Simple aggregation uses functions like .mean(), .median(), .min(), and .max() to summarize data. Grouping with .groupby() is more powerful because it lets me compare groups. For example, I can find the average trip length by age group, or the median age by user type. This helped me see how data science is not just about calculating one number, but comparing patterns between different categories.

Another big topic this week was continuous variables and distributions. I learned that a PDF shows the shape of a distribution, while a CDF shows the cumulative probability up to a certain value. I am starting to understand how to estimate probabilities from graphs, like estimating the probability that a penguin’s body mass is less than 5000 grams. I also learned that for a continuous variable, the probability of one exact value is 0, which felt strange at first but makes more sense now because probability is based on area under the curve.

One thing I still find a little confusing is visually estimating probabilities from density plots. I understand that the area under the curve represents probability, but it is hard to know exactly how much area is on one side of a value just by looking. The CDF feels easier to read because the y-axis directly gives the probability. I think I need more practice connecting PDFs and CDFs together.

One question I still have is when it is better to use value_counts() compared to groupby(). I understand that value_counts() is good for counting categories, and groupby() is good for calculating statistics by group, but sometimes they feel like they overlap. However, I feel like pandas is starting to make sense especially after doing the labs, but I still need more practice writing the expressions without constantly looking back at examples. 

Comments

Post a Comment

Popular posts from this blog

CST 334 - Week 3

Week 3 Summary: This week in CST 334 was a rollercoaster. We started working on PA3, which is all about building a memory allocator. Basically our own version of malloc() and free(). At first, I thought, “okay cool, pointers and structs again,” but this assignment humbled me quick. Managing memory manually, keeping track of chunks, splitting them, and then coalescing them back together sounds simple on paper but actually implementing it without causing segmentation faults or crashing everything was tough. One thing that stood out was just how fragile everything becomes when you're directly manipulating memory. A wrong pointer or forgetting to update a linked list will break the whole thing. A lot of my time was spent debugging and going through each line of my code just to figure out where things were going sideways. The split_node function specifically gave me a hard time. It took me a while to realize how important it is to keep the fwd and bwd pointers clean or everything falls ...
Read more

CST 334 - Week 2

Week 2 Summary:  This week in CST 334, we dug into how operating systems handle processes, and honestly, things are starting to make more sense now. I’ve heard the term “context switching” before, but I never really got what it meant until we broke it down. Learning the difference between voluntary and involuntary switches like when a process is just waiting for input vs. when the OS forces it to stop, helped a lot. It’s kind of wild to think about how much the OS is doing behind the scenes just to keep everything running without us even noticing.  We also got into how processes are created in C with fork(), exec(), and wait(). The fork() function really threw me off at first. I was like is the same program just running twice? But it made sense after printing out a bunch of values and looking at the return codes. It’s pretty cool how the parent and child processes end up doing different things, even though they start from the same place. Later in the week, we went over schedul...
Read more

CST 370-30 - Algorithm Design & Analysis Week 1

 CST 370-30 - Algorithm Design & Analysis Week 1 This week in CST 370, we went over the basics of algorithm design and analysis and what to expect from the course. A big focus was on understanding how to think about problems in terms of best case and worst case scenarios, not just getting an answer. We worked on logic puzzles like the fake coin and ball matching problems, which helped me see how algorithms can be about strategy and guarantees rather than code. These problems showed how careful reasoning can minimize steps while still covering all cases, which I think will be important later in the class.
Read more