Data Science

PYTHON PROGRAMMING

Public versus Private – Say versus Think

Generally, in programming, when something is public, you can access it and use it; when it’s private, you cannot. It’s like thinking something versus saying something: When you think something, it remains yours; but whatever you say out loud stops being only yours and becomes public.

Things work differently in Python. You probably heard that in Python nothing is really private. What does this mean? Does Python have private attributes and methods¹?

We use these two terms, public and private, in the context of Python classes’ methods and attributes. When an attribute is private, you should not use it; when a method is private, you should not call it. You perhaps noticed I used the world "should". This is because, as I already mentioned, things work differently in Python: When something is public, you can access and use it; when it’s private, you should not do it – but this does not mean you can’t. So, when you think something in Python, it should remain yours – but anyone can hear it anyway, using rather simple measures.

As you see, Python is not strict in terms of privacy. It recommends you to follow some rules, not makes you to follow them. It recommends users of a class to not access private methods and attributes – but the users can do it anyway, and what’s more, they don’t have to put too much effort into this.

In this article, I am going to explain these things in simple words, using simple examples.

When you think something, it remains yours; but whatever you say out loud stops being only yours and becomes public.

When you think something in Python, it should remain yours – but anyone can hear it anyway, using rather simple measures.

"Private" methods and attributes

There is no real privacy in Python. What Python offers is pseudo-privacy, or quasi-privacy. It has two levels, which I call indication privacy and hide-and-seek privacy.

Indication privacy

You can indicate that a particular attribute is private. To do it, enough to add a leading underscore to its name. Doing so, you indicate, or suggest, or recommend, that the method/attribute should be treated as private, meaning it should not be used outside the class.

Thus, instance.do_it() is a regular (public) method while instance._do_it() is a method indicated as private. Hence, as the class’s user, you’re asked not to use it. It’s there because it serves some implementational purposes – and you have nothing to do with it. It’s not a secret. You can have a look at it, no one hides anything from you. But it’s not for you. Take what you’re offered and get your hands off what you’re not.

Let’s consider a simple example:

# class_me.py
class Me:
 def __init__(self, name, smile=":-D"):
 self.name = name
 self.smile = smile
 self._thoughts = []

 def say(self, what):
 return str(what)

 def smile_to(self, whom):
 return f"{self.smile} → {whom}"

 def _think(self, what):
 self._thoughts += [what]

 def _smile_to_myself(self):
 return f"{self.smile} → {self.name}"

(If you don’t know why I wrote self._thoughts += [what] and not self._thoughts += what, visit Appendix 1.)

Okay, so we have a class Me, which represents, well, you – at least when you created it. It has the following attributes:

• .name, a public attribute → your name is definitely public.
• .smile, a public attribute → your smile is visible outside, so it’s definitely public.
• ._thoughts, a private attribute → your thoughts are definitely private, aren’t they?

As you see, the two public attributes’ names do not have the leading underscore, and the only private attribute’s name does.

Now let’s have a look at the available methods:

• .say(), a public method → when you say something, people can hear you, so your words are public.
• .smile_to(), a public method → when you smile to someone, this person and the people around can see you smiling.
• ._smile_to_myself(), a private method → this is a different kind of smile; it’s reserved for the class’s author (in our example, for you), and it’s done when no one is looking – that’s why it’s a private method.
• ._think(), a private method → when you think something, it’s your private thought; if you want to say it out loud, you should use the public .say() method.

Let’s play with the class. I will create the class’s instance for myself, so I’m going to call it marcin. You can create an instance for yourself.

>>> from class_me import Me
>>> marcin = Me(name="Marcin")
>>> marcin # doctest: +ELLIPSIS
<__main__.Me object at 0x...>
>>> marcin.say("What a beautiful day!")
'What a beautiful day!'
>>> marcin.smile_to("Justyna")
':-D → Justyna'

I used doctest to format code in the block above. It helped me ensure that the code was correct. You can read more about this documentation testing framework from the following article:

Python Documentation Testing with doctest: The Easy Way

If you’d like to copy and paste the code as doctest and run it yourself that way, visit Appendix 2 at the end of the article, which contains the remaining code formatted that way (e.g., the Me class’s code).

Okay, all looks fine and dandy. So far, however, we’ve been polite and haven’t even looked at the private methods and attributes; we’ve only used the public ones. It’s time to go a little naughty:

>>> dir(marcin) #doctest: +NORMALIZE_WHITESPACE
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', 
'__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', 
'__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', 
'__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', 
'__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 
'__weakref__', '_smile_to_myself', '_think', '_thoughts', 'name', 
'say', 'smile', 'smile_to']

What do we see? Actually, everything. We certainly see public attributes .name and .smile and public methods .say() and .smile_to(). But we do also see private attribute ._thoughts and private methods ._think() and ._smile_to_myself(). In addition, we see many more methods and attributes that we didn’t create.

Remember that a method named using the .__name__() convention is a dunder method, not a private one. We’ll talk about this some other time.

Since we were able to see the private attributes, it’s likely we will be able to use them, too:

>>> marcin._think("My wife is so beautiful!")
>>> marcin._think("But let this be my secret!")

Nothing happened? Then maybe it’s fine? Maybe we can use private methods, but whatever they’re doing is hidden from us?

Of course not. It’s just that the ._think() method does not return anything (or rather returns None) but saves thoughts to the ._thoughts attribute, also private. Let’s check if you can see my private thoughts:

>>> marcin._thoughts
['My wife is so beautiful!', 'But let this be my secret!']

Yes, you can. A last test: let’s check if you can see me smiling to myself:

>>> marcin._smile_to_myself()
':-D → Marcin'

You can, too. So, you clearly can see private attributes, and you can use private methods – even though I specifically indicated, by adding the leading underscore to these attributes’ and methods’ names, that they were private and so I did not want you to use them. Using private methods or attributes is be a bit like spying on me in the shower – you can see what I want to hide from you.

Sometimes, however, for this or another reason, you may want to hack an existing class; this could mean overwriting a private attribute or method. And this is where Python’s approach shines. Theoretically, these attributes are private so you should not use them; sometimes, using them can even break a class. It’s also a sort of safeguard; you know these attributes are private so it’s better not to touch them.

But when you know what you’re doing, when your purpose requires you to use private attributes – Python makes this possible. This opens lots of additional opportunities for Python developers.

Using private methods or attributes is a bit like spying on me in the shower – you can see what I want to hide from you.

This opens lots of additional opportunities for Python developers.

A little exaggerating, in Python you can do whatever you want. You can overwrite built-in functions, exceptions and the like. (If you want to learn more about overwriting exceptions, read this Better Programming article.) And you can use private attributes. That’s fine, assuming – like in the case of any code – that you do not want to do any harm to the user’s computer.

I am sure you will agree that this sort of privacy is weak, as the user can use such private attributes and classes just like public attributes and classes. Python, however, offers a more strict approach to privacy, which I call hide-and-seek privacy.

Hide-and-seek privacy

While the indication level of privacy only consisted of indicating whether an attribute is private or public, the hide-and-seek level goes further. As you will see in a second, it helps you, to some extent, protect private attributes.

Does this mean that this time, private attributes and methods will be truly hidden, and that the user will not be able to use them? Not entirely. As I wrote, hide-and-seek privacy offers some level of protection – but not full protection. Python does so thanks to a method called name mangling.

When you want to use name mangling, and so hide-and-seek privacy, you need to add not one but two leading underscores to private attributes’ names. In our Me class, this would be, for instance, .__thoughts and .__think(). Thanks to name mangling, private attributes or methods are modified in a particular way so that it is harder to access them from outside the class.

Let’s see this in work. We will revise our Me class first; let’s change its name to PrivateMe (see Appendix 2 for the code formatted for doctesting):

# class_me.py
class PrivateMe:
 def __init__(self, name, smile=":-D"):
 self.name = name
 self.smile = smile
 self.__thoughts = []

 def say(self, what):
 return str(what)

 def smile_to(self, whom):
 return f"{self.smile} → {whom}"

 def __think(self, what):
 self.__thoughts += [what]

 def __smile_to_myself(self):
 return f"{self.smile} → {self.name}"

First, let’s create an instance – again, this will be an instance of me – and use the public methods:

>>> marcin = PrivateMe(name="Marcin")
>>> marcin.say("What a beautiful day!")
'What a beautiful day!'
>>> marcin.smile_to("Justyna")
':-D → Justyna'

(If you’re wondering whether Justyna is my wife or I’m smiling to another girl, you can rest assured; she is!)

So far so good, but this does not come as a surprise – after all, we’ve used public methods. Previously, we succeeded to use also private methods, like ._smile_to_myself(). Let’s try whether we succeed this time, too. In order to check this out, I’ll try to smile to myself using the .__smile_to_myself() method:

>>> marcin.__smile_to_myself()
Traceback (most recent call last):
 ...
AttributeError: 'PrivateMe' object has no attribute '__smile_to_myself'

Ha! We know the PrivateMe class has the __smile_to_myself() method, but it’s not available for us to use. Clearly, it’s protected, as any private method should.

Nonetheless… It looks as though the method was fully protected while not that long ago I claimed that in Python, private attributes are not fully protected. So, what’s going on?

We’ve just experienced how name mangling works. It hides private attributes – or rather, however strange that might sound, it hides the names of private attributes. In other words, it changes their names in a particular way; the new name will follow the following _ClassName__attribute notation:

class MyClass:
 __privacy = None # this becomes ._MyClass__privacy 
 def __hide_me(self): # this becomes ._MyClass__hide_me()
 pass

That way, you cannot access an attribute using its original name, but you can access it using the name updated by name mangling. In our PrivateMe class , this would work like this:

>>> marcin._PrivateMe__smile_to_myself()
':-D → Marcin'

You can see for yourself that this attribute is there, just renamed. We will, certainly, see this in the output of the dir() function:

>>> dir(marcin) # doctest: +NORMALIZE_WHITESPACE
['_PrivateMe__smile_to_myself', '_PrivateMe__think',
 '_PrivateMe__thoughts', '__class__', '__delattr__',
 '__dict__', '__dir__', '__doc__', '__eq__', '__format__',
 '__ge__', '__getattribute__', '__gt__', '__hash__',
 '__init__', '__init_subclass__', '__le__', '__lt__',
 '__module__', '__ne__', '__new__', '__reduce__',
 '__reduce_ex__', '__repr__', '__setattr__',
 '__sizeof__', '__str__', '__subclasshook__',
 '__weakref__', 'name', 'say', 'smile', 'smile_to']

Our private methods and attributes are available using new names:

• .__smile_to_myself() → ._PrivateMe__smile_to_myself()
• .__think() → ._PrivateMe__think()
• .__thoughts → ._PrivateMe__thoughts

Name mangling² has enabled us to achieve the hide-and-seek level of privacy.

Remember one more thing. When you want to make an attribute private by adding two leading underscores, do not add two additional underscores at the end of the name. A method named like that becomes a so-called dunder (_double-under_score) method – and these are definitely not private; in fact, they are opposite to private. We’ll talk about them some other time. To use name mangling, it’s enough for you to remember this naming rule: don’t use the .__name__() convention for private methods, as this will not work.

Conclusion

We’ve discussed the concept of privacy in the context of object-oriented programming in Python. When writing a class, you may sometimes wish to hide some implementation details, and you can achieve that by making some the class’s attributes and methods private. But they are never truly private.

This approach does not sound natural to me. When I think about a private attribute, I imagine it as an attribute that cannot be seen and used outside the class. Along the same lines, it is a public attribute that can be seen and used that way.

If your imagination works in a similar way, you need to use the world-changing glasses so that you can move around the Python world without falling over every now and then. You have to wear these glasses every time you use Python. Sooner or later, they will help you get used to the different world of Python, one in which the concept of privacy works so differently.

You need to use the world-changing glasses so that you can move around the Python world without falling over every now and then.

Sooner or later, they will help you get used to the different world of Python, one in which the concept of privacy works so differently.

In summary, Python does not enable you to fully protect a class’s attribute. It offers, however, two levels of protection, which I called the indication and hide-and-seek privacy.

Indication privacy. You can indicate an attribute as private and believe that no one will use this attribute outside the class. The indication method is based on trust: we trust that the class’s users will not use its private attributes. The approach uses no protection measures besides that.

The indication method is based on trust: we trust that the class’s users will not use its private attributes. The approach uses no protection measures besides that.

Hide-and-seek privacy. This is a higher level of privacy – and most we can get from Python in terms of the privacy of a class’s attributes. While in the case of indication privacy you can use attributes indicated as private the same way as public ones, here you cannot. You’re offered a certain level of protection of private attributes. It’s still not full protection; private attributes are hidden thanks to changed names. You still can find, access and use them – but at least they are somehow protected. They are not really hidden, as dir() will show us all the class attributes, including public and private ones, but the latter will have changed names.

---

Thanks for reading this article. I hope privacy in the context of Python classes will no longer constitute a problem for you. While at first glance this topic may seem difficult, or at the very least peculiar, you’ll quickly get used to the strange world of Python privacy. Rest assured that many Python developers appreciate how these things work in Python. If you don’t, chances are you’ll join them sooner or later.

As for me, not only have I nothing against how Python treats privacy, I even appreciate it. Quite a few times have I used this approach, and it’s good to know that it’s there, just in case, waiting for me to spy on classes’ attributes and methods.

If you enjoyed this article, you may also enjoy other articles I wrote; you will find them here. And if you want to join Medium, please use my referral link below:

Join Medium with my referral link – Marcin Kozak

Footnotes

¹ Remember that in Python, methods are attributes of the class. Thus, whenever I mention the privacy of attributes, I mean the privacy of attributes, including methods.

² Name mangling serves two purposes:

• It increases the level of protection of the class’s private attributes and methods.
• It ensures that private attributes from the parent class are not overwritten by a class inheriting from it. So, when you use two leading underscores, you do not have to worry that this attribute in the class be overwritten by the inheriting class.

This article is about the first point. The second one is beyond the scope of this article, however; we’ll discuss it some other time.

Appendix 1

This appendix explains why, when coding the Me class, I wrote

self._thoughts += [what]

and not

self._thoughts += what

The in-place concatenation, +=, works in the following way:

>>> x = [1, 2, 3]
>>> y = [4, 5, 6]
>>> x += y
>>> y
[4, 5, 6]
>>> x
[1, 2, 3, 4, 5, 6]

As you see, this operation adds two lists; as in-place operation, it affects the first of them and leaves the second unchanged. However, this does not work for non-iterable objects, like numbers (here, int):

>>> x += 5
Traceback (most recent call last):
 ...
TypeError: 'int' object is not iterable

So, you can use in-place concatenation to add to a list another iterable, like a list, a tuple, a range object, and a generator:

>>> x += (10, 20)
>>> x
[1, 2, 3, 4, 5, 6, 10, 20]
>>> x += range(3)
>>> x
[1, 2, 3, 4, 5, 6, 10, 20, 0, 1, 2]
>>> x += (i**2 for i in range(3))
>>> x
[1, 2, 3, 4, 5, 6, 10, 20, 0, 1, 2, 0, 1, 4]

Strings are also iterables, so you can add them to a list, too:

>>> x += "Cuma"
>>> x
[1, 2, 3, 4, 5, 6, 10, 20, 0, 1, 2, 0, 1, 4, 'C', 'u', 'm', 'a']

As you see, the "Cuma" string was treated as an iterable of its individual characters, and it is them – these characters – that were added to x, not the word itself.

This is why self._thoughts += what would not work. Had we used it, we would have achieved the following undesirable effect:

>>> marcin._think("I am tired.")
>>> marcin._thoughts
['I', ' ', 'a', 'm', ' ', 't', 'i', 'r', 'e', 'd', '.']

Hence, we need to add the thought to ._thoughts as the element of a list, that is, [what]. This single-element list is the iterable to be added to ._thoughts.

Appendix 2

Class Me formatted for doctest:

>>> class Me:
... def __init__(self, name, smile=":-D"):
... self.name = name
... self.smile = smile
... self._thoughts = []
... def say(self, what):
... return str(what)
... def smile_to(self, whom):
... return f"{self.smile} → {whom}"
... def _think(self, what):
... self._thoughts += [what]
... def _smile_to_myself(self):
... return f"{self.smile} → {self.name}"

Class PrivateMe formatted for doctest:

>>> class PrivateMe:
... def __init__(self, name, smile=":-D"):
... self.name = name
... self.smile = smile
... self.__thoughts = []
... def say(self, what):
... return str(what)
... def smile_to(self, whom):
... return f"{self.smile} → {whom}"
... def __think(self, what):
... self.__thoughts += [what]
... def __smile_to_myself(self):
... return f"{self.smile} → {self.name}"

Share This Article

• Share on Facebook
• Share on LinkedIn
• Share on X