python►c-pandas

Python-c05-Filtering 

1
2
3
4
5
people = {
    "first": ["Corey", 'Jane', 'John'], 
    "last": ["Schafer", 'Doe', 'Doe'], 
    "email": ["CoreyMSchafer@gmail.com", 'JaneDoe@email.com', 'JohnDoe@email.com']
}
1
import pandas as pd
1
df = pd.DataFrame(people)
1
df

first last email
0 Corey Schafer CoreyMSchafer@gmail.com
1 Jane Doe JaneDoe@email.com
2 John Doe JohnDoe@email.com 
1
filt = (df['last'] == 'Doe')
1
df[filt]

first last email
1 Jane Doe JaneDoe@email.com
2 John Doe JohnDoe@email.com 
1
2
# filter df with filt and return just the column first and email
df.loc[filt, ['first', 'email']]

first email
1 Jane JaneDoe@email.com
2 John JohnDoe@email.com 
1
2
# not in filter ~
df.loc[~filt, 'email']

0 CoreyMSchafer@gmail.com
Name: email, dtype: object

1
2
3
4
# isin filter
lasts = ['Doe', 'Mike', 'Bob']
filt = df['last'].isin(lasts)
df.loc[filt, 'email']

1 JaneDoe@email.com
2 JohnDoe@email.com
Name: email, dtype: object

1
df = pd.read_csv('data/survey_results_public.csv')
1
df

Respondent Professional ProgramHobby Country University EmploymentStatus FormalEducation MajorUndergrad HomeRemote CompanySize ... StackOverflowMakeMoney Gender HighestEducationParents Race SurveyLong QuestionsInteresting QuestionsConfusing InterestedAnswers Salary ExpectedSalary
0 1 Student Yes, both United States No Not employed, and not looking for work Secondary school NaN NaN NaN ... Strongly disagree Male High school White or of European descent Strongly disagree Strongly agree Disagree Strongly agree NaN NaN
1 2 Student Yes, both United Kingdom Yes, full-time Employed part-time Some college/university study without earning ... Computer science or software engineering More than half, but not all, the time 20 to 99 employees ... Strongly disagree Male A master's degree White or of European descent Somewhat agree Somewhat agree Disagree Strongly agree NaN 37500.0
2 3 Professional developer Yes, both United Kingdom No Employed full-time Bachelor's degree Computer science or software engineering Less than half the time, but at least one day ... 10,000 or more employees ... Disagree Male A professional degree White or of European descent Somewhat agree Agree Disagree Agree 113750.0 NaN
3 4 Professional non-developer who sometimes write... Yes, both United States No Employed full-time Doctoral degree A non-computer-focused engineering discipline Less than half the time, but at least one day ... 10,000 or more employees ... Disagree Male A doctoral degree White or of European descent Agree Agree Somewhat agree Strongly agree NaN NaN
4 5 Professional developer Yes, I program as a hobby Switzerland No Employed full-time Master's degree Computer science or software engineering Never 10 to 19 employees ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
19097 19098 Professional developer Yes, I program as a hobby Canada No Employed full-time Bachelor's degree A business discipline A few days each month 10 to 19 employees ... Disagree Male Some college/university study, no bachelor's d... White or of European descent Somewhat agree Agree Disagree Agree NaN NaN
19098 19099 Student Yes, I program as a hobby India No Not employed, and not looking for work Secondary school NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
19099 19100 Professional non-developer who sometimes write... Yes, I program as a hobby United Kingdom No Independent contractor, freelancer, or self-em... Bachelor's degree Computer science or software engineering Never NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
19100 19101 Professional developer Yes, I program as a hobby United States No Employed full-time Some college/university study without earning ... A humanities discipline Less than half the time, but at least one day ... 100 to 499 employees ... Disagree Male Some college/university study, no bachelor's d... White or of European descent Somewhat agree Somewhat agree Disagree Agree 110000.0 NaN
19101 19102 Professional developer Yes, I program as a hobby France No Employed full-time Master's degree Computer science or software engineering All or almost all the time (I'm full-time remote) 100 to 499 employees ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

19102 rows × 154 columns

1
2
# na=False means if not found the result, put False
filt = df['MajorUndergrad'].str.contains('software', na=False)
1
df.loc[filt]

Respondent Professional ProgramHobby Country University EmploymentStatus FormalEducation MajorUndergrad HomeRemote CompanySize ... StackOverflowMakeMoney Gender HighestEducationParents Race SurveyLong QuestionsInteresting QuestionsConfusing InterestedAnswers Salary ExpectedSalary
1 2 Student Yes, both United Kingdom Yes, full-time Employed part-time Some college/university study without earning ... Computer science or software engineering More than half, but not all, the time 20 to 99 employees ... Strongly disagree Male A master's degree White or of European descent Somewhat agree Somewhat agree Disagree Strongly agree NaN 37500.0
2 3 Professional developer Yes, both United Kingdom No Employed full-time Bachelor's degree Computer science or software engineering Less than half the time, but at least one day ... 10,000 or more employees ... Disagree Male A professional degree White or of European descent Somewhat agree Agree Disagree Agree 113750.0 NaN
4 5 Professional developer Yes, I program as a hobby Switzerland No Employed full-time Master's degree Computer science or software engineering Never 10 to 19 employees ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
7 8 Professional developer Yes, both Poland No Employed full-time Master's degree Computer science or software engineering All or almost all the time (I'm full-time remote) Fewer than 10 employees ... Somewhat agree Male A master's degree White or of European descent Agree Somewhat agree Disagree Agree NaN NaN
8 9 Professional developer Yes, I program as a hobby Colombia Yes, part-time Employed full-time Bachelor's degree Computer science or software engineering Less than half the time, but at least one day ... 5,000 to 9,999 employees ... Strongly disagree Male A bachelor's degree Hispanic or Latino/Latina Somewhat agree Strongly agree Disagree Strongly agree NaN NaN
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
19091 19092 Professional developer No Sweden No Employed full-time Master's degree Computer science or software engineering Never 100 to 499 employees ... Strongly disagree Male A master's degree White or of European descent Disagree Agree Somewhat agree Strongly agree NaN NaN
19095 19096 Professional developer Yes, both United States No Employed full-time Bachelor's degree Computer science or software engineering A few days each month 100 to 499 employees ... Disagree Male A bachelor's degree White or of European descent Somewhat agree Somewhat agree Disagree Somewhat agree NaN NaN
19096 19097 Professional developer No France No Employed full-time Master's degree Computer science or software engineering Never 20 to 99 employees ... Strongly disagree Male High school White or of European descent Strongly agree Somewhat agree Somewhat agree Somewhat agree NaN NaN
19099 19100 Professional non-developer who sometimes write... Yes, I program as a hobby United Kingdom No Independent contractor, freelancer, or self-em... Bachelor's degree Computer science or software engineering Never NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
19101 19102 Professional developer Yes, I program as a hobby France No Employed full-time Master's degree Computer science or software engineering All or almost all the time (I'm full-time remote) 100 to 499 employees ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

8098 rows × 154 columns