How to screen Python developers’ skills

If you are looking for a person with Python developer skills, know that finding them is only half the journey. So even if you can source them, it might take some extra effort to truly understand what Python skills really are. The real Python skill set is a complex thing to understand, but here we introduce an expert in the field, Wojtek Erbetowski to bring the topic to light.

1. What is Python?

Python is one of the most popular programming languages (ranking second according to the TIOBE Index as of September 2021). It was first released in 1991 and gained its popularity for being simple to learn, yet powerful in solving problems. The syntax is similar to the C++ family, yet is equipped in a variety of modern solutions.

The key advantages of Python are:

• Flat learning curve. It brings in users with little technical experience, like academics for research, or business for automation.
• It’s owned by an independent Python Software Foundation (PSF) and guided by the community.
• Powerful scientific tools (e.g. SciPy).
• Good documentation and community support. This includes many local user groups and conferences.

2. What is important for an IT Recruiter to know about Python?

Since Python has a variety of uses. It is often difficult to distinguish candidates from academics, through network programming and web applications to simple scripting. Yet it is often easier for a developer to learn new Python than to change her domain (e.g. Data Science to Web Applications).

In 2007, Python 3.0 was released. It was rejected by the community due to backward incompatibility. That said, in 2020, 94% of Python users switched to Python 3.

Python is dynamically typed, which ease scripting and writing small applications. It is often said to bring difficulties in building large and long-lasting applications.

3. How to verify Python developer skills at the screening phase?

Screening is tiring, arduous, and time-consuming. Usually IT recruiters turn to resume screening, technical screening (coding tests), and interviews to assess Python skills.

Below you find some useful and practical recruiting tips how to verify Python skills using each method.

3.1. Technical screening of Python developer skills based on CV

Fortunately (for recruiting), Python has fewer libraries and technologies to screen for, than e.g. Java. Yet a number of things might be confusing.

Python language	Popular version 3.7, 3.8, 3.9, 3.10
Python web frameworks	Django, Flask, Bottle, Tornado, web2py, FastAPI
ORM	SQLAlchemy, Django ORM, peewee
Build tools	Setup-tools, pip, tox
Test frameworks	UnitTest, py.test
Asyncio	Python 3.5, Tornado, Twisted
Data analysis tools	NumPy, SciPy, Pandas, PySpark

Highlights:

• Python 2 and 3 are very different. Python 2.7 is the latest (and last) version of Python 2. Python 3 version is evolving and the latest version is 3.10
• Asyncio is only met in Python 3.
• Tornado, Twisted, Gevent are all libraries for async programming.
• Django is by far the most popular web framework.
• Unit test library is built into the language, therefore rarely mentioned separately.

Certification

Python Community does not provide any Certification program. Therefore it is rare to meet any (independent) certificates for Python language or libraries.

Full-stack developers

Fullstack often means that a Python developer may also create HTML pages with CSS and JavaScript. Frontend skills are welcome and most of JavaScript screening knowledge applies here.

3.2. Technical screening of Python technical skills during a phone/video interview

As it is pretty difficult to draw reliable conclusions about candidate’s programming skills from resume itself, IT recruiters often conduct a phone or video interview. Such an interview gives an opportunity to ask questions concerning their Python skills. In many companies such a technical phone or video interview is a basis for making a decision whether to invite somebody for on-site, technical interview with IT team or not, so it’s worth to choose and ask the right technical questions.

As questions themselves can vary based on the position you are recruiting for and specific skills you want to verify, based on my experience I would recommend following general questions to find out more about candidate’s Python skills:

A. Python interview question concerning developer’s experience

• Q1: With which technologies listed in your CV, did you have commercial experience in the past 2 years? What were your responsibilities? What was your biggest achievement?
• Why ask Q1: By asking this question, you can find out more about candidate’s Python up-to-date experience, responsibilities, and achievements. Thus you get to know your candidate better.

B. Python interview question concerning developer’s knowledge and opinions

• Q2: Which Python open source libraries do you consider to be valuable and why?
• Why ask Q2: This question lets the candidate demonstrate their knowledge of Python ecosystem. Moreover, the candidate can point out what he takes into account when choosing a certain technology. As follow-up questions, you can also ask what in his opinion advantages of commercial solutions over open source and vice versa are. This shows how the candidate thinks.

• Q3: I’ve noticed you listed framework X on your CV. What’s your opinion about it? Is it a good choice?
• Why ask Q3: Such an open question allows a candidate to demonstrate their knowledge if they have it. You can also play devil’s advocate based on their answers “But why not use framework Y for that?”. Thus you can see how the candidates choose a technology, justify their choices and whether his opinions are in line with your IT teams.

C. Python interview question concerning developer’s decision making regarding Python technology

• Q4: On what stage did you join recent projects? Were you involved in the choice of technology or project setup? If yes, which technology did you choose or recommend for the project and why?
• Why ask Q4: Such a questions shows not only how a candidate makes choices regarding technology but also whether they were included in making decisions. Especially important if you are looking for someone with experience.

D. Behavioral question for python developer

• Q5: Think about programming project decision you made that was a failure. Why do you think it was a mistake? Why did it happen? Could there be anything done differently in order to turn it into success? What steps did you take to improve the situation? What did you learn from this experience?
• Why ask Q5: That type of question will help you find out how the candidate perceives their decisions, draws conclusions and how much they learn from their previous experience.

3.3. Technical screening of Python skills with online coding test

Online coding tests can be a great way to screen Python skills. Below you will find information how you can use programming tests and which ones to choose.

Online Python coding tests – the primary or additional method for screening

Some companies use programming test as an addition to screening candidates after the resume is analysed and phone or video interview is conducted. But more and more companies turn to coding tests as the first method of screening programmers. Companies nowadays tend to invite all candidates that apply for a certain position to take a test. This way, recruiters only spend their time on candidates that passed the exam, proving that they have Python programming skills. On the other hand, candidates aren’t asked to dedicate too much of their time to the recruitment process early on – instead, they quickly complete the test (usually within an hour) at their own convenience both in terms of time and place.

Screen Python Developer skills with a built-in PyCharm IDE

At DevSkiller, we continue to push the boundaries when it comes to developer testing.

Our latest innovation is to let candidates solve our Python coding tests using an in-browser PyCharm IDE from JetBrains. No downloads, no waiting and no fuss.

Candidates save time during testing by not having to wait for dependencies to install or for programs to load. They can get straight to work focusing on their solutions to our Python coding tests instead of learning how to make a primitive IDE they’re unfamiliar with, work in their favor.

Candidates no longer have to clone the code – they can literally start coding as soon as they open their test invite. Candidates can run unit tests, preview and play their solutions, just like in their regular work.

For your company, this means offering a better candidate experience than ever before. It also means lower candidate drop-off during the process. The faster the process, the more likely it is for your candidates to see it through. It means you won’t let the best candidates slip through your fingers.

Python online coding test – which one to choose?

There are a lot of Python online coding tests on the market, but usually, they verify a candidate’s knowledge rather than programming skills and abilities to solve real-life coding problems. That’s why a lot of experienced programmers don’t like that kind of assessment and, what’s more, they aren’t able to get a good score as they cannot show their experience and skills in solving real-life programming problems.

So what features should a good coding test have? Below you will find my top 5 features you should be looking for:

1. Python programming tests should verify not only Python skills as far as technology in general, but also frameworks and libraries;
2. Python coding tests should consist of a couple of tasks, not only multiple-choice tests, but above all, programming tasks which difficulty should match the responsibilities the candidates will have if they get the job (tests shouldn’t be based on algorithmic tasks as thus you cannot verify whether a candidate can solve real-life coding challenges);
3. Preferably, Python programming tasks should be prepared on the company’s own code base, as this is the best way to imitate the challenges the candidate will have once they get the job;
4. There is no point in forbidding the programmer to use external sources. The best way is to create an environment similar to the one they would have at work, so an open book exam is much more appropriate. Moreover, the candidate should be able to use their preferred IDE, just as he does on a daily basis.
5. Coding tests can also include a task with code review challenges to test Python skills. This gives very accurate information about the experience of the candidate, their approach to code quality and efficiency of debugging and finding traps. Such an assessment should be obligatory when recruiting Python senior developers.

Editor’s note: This post was originally published February 28, 2016. It has been extensively updated to represent the latest technical information and best practices.

Author – Wojtek Erbetowski

Wojtek is a hands-on leader with ~10 years of APIs / web apps experience in Python and Java. Worked as Head of Engineering @ Polidea, CTO @ GrowBots and actively participates as a Community Leader @ Warsaw Java User Group and Mobile Warsaw and Organizer of the MCE Conference. Open source library contributor, software developer, agile advocate. He was working in several business areas, e.g. sales, old-fashioned and new media advertising, e-mail marketing, pharmaceutical corp, EU financed portals, wholesale bookstore and more) and turning business needs into specialized apps.

DevSkiller ready to use online Python coding assessment tests

DevSkiller helps companies in IT recruitment process by providing them with an infrastructure which lets them test programming skills of candidates in the environment which imitates 1st day at work experience. We allow companies to use their own code base, whole IT projects using frameworks and libraries, to test programmers online. However, we also have some ready-to-use Python coding tests which you can use in your technical recruitment campaign.