I am writing an object-oriented app to help our developers manage some cloud systems. I’d like to make the configuration information available to all the classes, but I’m not sure of a good way to do that. Everything I can think of seems to fall under the category of “global variables” which as far as I know is a Very Bad Thing.
I already have a logging Mixin class that enables logging for every class that inherits it, and I was wondering if that’s the right way to approach the configuration data:
class LoggingMixin:
@classmethod
@property
def log(cls):
return logging.getLogger(cls.__name__)
class TestClassA(LoggingMixin):
def testmethod1(self):
self.log.debug("debug message from test class A")
if __name__ == "__main__":
logging.basicConfig(
format="{created:<f} {levelname:>5s} {name}.{funcName:>8s} |{message}|",
level=logging.DEBUG,
style="{",
)
a = TestClassA()
a.testmethod1()
Outputs (in case you are curious)
1688494741.449282 DEBUG TestClassA.testmethod1 |debug message from test class A|
What’s a good way of making data from a class available to all classes/objects? It wouldn’t be static, it’d be combined from a JSON file and any command line parameters.
If I copied the example above but changed it to a ConfigMixin, would that work? With the logging example, each class creates its own logger object when it first calls self.log.debug, so that might not work because each object needs to get the same config data.
Is there a pattern or other design that could help? How do you make configuration data available to your whole app? Do you have a config object that can get/set values and saves to disk, etc?
Thank you for reading, my apologies for poorly worded questions.
I use a singleton class called Config (some times I use it as a Cache too)
Singleton pattern: https://www.geeksforgeeks.org/singleton-pattern-in-python-a-complete-guide/
Config class using Singleton pattern: https://charlesreid1.github.io/a-singleton-configuration-class-in-python.html
I don’t see why you need a singleton, just use use a global variable if you really need one. A singleton has all the same downsides but just hides them by not looking like a global.
https://nedbatchelder.com/blog/202204/singleton_is_a_bad_idea.html
You can use global variables, it is true, but I prefer singleton because I have all config variables and logic encapsulated in a class. The first time the singleton object is created it reads all the config variables from a file, and with its methods get and set manages them. The Config class doesn’t know the parameters names of the config file neither the number, it dynamically reads the file and creates the attributes, so It is a very reusable code that when you program using OOP is a more natural way than implementing like a module.
Related to this, I group config variables by sections so it is more clearly for me, then the singleton object dynamically creates an array for each section, and a variable into the array for every variable in this section of the config file. Access to the config info is as easy like this:
conf = Config() conf.get('DB', 'server') conf.get('DB', 'login') conf.get('DB', 'password')
This is great, thank you! I’ll look at implementing this soon.
In programming, people often talk in absolutes, but there‘s often exceptions to these rules, depending on situation and depending on the programming language. In Python, things like dependency injection don‘t make a lot of sense, because it just isn‘t needed. It‘s a pattern extensively used in other languages like Java, but only because Java has limitations that don‘t exist in Python. Trying to use Java patterns into Python just because it „sounds right“ or because it‘s a „best practice“ in another language does not make your code actually good python. It‘s not necessarily bad, but more complicated than it needs to be, and simpler is usually better if it yields the same end result. Don‘t overthink it.
You say global variables are a very bad thing, but do you know why that is? People often hear about things like that but don‘t learn about the „why“, which hinders you from properly thinking about your code.
Global variables are a bad thing because it is very hard to reason about their state at certain points in your programs execution. But what if your variable never changes, like a constant? Then you don‘t really have that problem. So an immutable config read in at startup and accessed by multiple classes/functions works just fine. You could also have a function that, when you run it the first time loads the config, „caches“ it in a global variable and then in subsequent calls just returns the global config.
To ensure immutability, you can use a frozen dataclass or something with validation built in like frozen pydantic models.
If you think you need mutability, think again. It’s better to put whatever you need to mutate outside of the global Config class, and then initialize that with the config. But keep the config itself read-only.
Agree.
Globals are a big problem in multithreaded / shared memory envs even when designs have a clear “here is where globals are set, and this is when”.
If single threaded, it’s a style choice but can lead to subtle bugs when the global keyword is forgotten - for ex, a function is accessing a global variable without the global keyword.
Then, later on, a developer updates the function to store a result in the variable, which converts the variable to local. Suddenly code which has been working “for years” may suddenly be askew and it’s not necessarily clear why (a good reason to lint code imo).
So if using global, be diligent in identifying them as such! Have a style standard (g_<var name> to help.
This is probably a good point to add that I do try to generally avoid globals as well. I am a fan of limiting mutable state in general. But for constants (whether they are hard coded in the source code or read from a file doesn’t really matter to me) they make sense.
And just to emphasize again, if you are building an App with user settings that may be changed during runtime, the „global constant config“ pattern does not apply. It only really works for static configuration read at startup.
For a mutable runtime config, you really have to think hard about when it is allowed to change and how these changes are propagated throughout your app, otherwise you may end up with inconsistent behavior where part of your program still uses the old values. There’s many ways to solve this depending on the programs architecture and requirements though.
Ah. Great point. In a threaded environment with dynamic configuration, paying attention to the exact use case is essential! Hidden constraints and unknown requirements might undo generic advice. That said, publisher/subscriber is a decent async pattern!
Imo for configs it’s fine having a config file that’s accessed via static methods or a singleton. I’m open to any good arguments against that practice though.
You need traitlets yesterday. It comes with a logging attribute and has extensive support for configuration - via command-line, or config files or python oop.
Hey, I implemented the config system using traitlets and it works great! Thank you again!
Ok now those look really useful! You are an excellent person and I appreciate you.
You could use dependency injection to set up config once and inject it where needed. at work we use https://pypi.org/project/inject/ . It has its quirks with the autoinjection, you sometimes need to tell it explicitly which parameters to inject, but works great overall.
You have the right idea.
One approach is to store cfg in json w/a class that reads that json. Remember to validate the json vs schema first!
That cfg class should also set env vars so that you don’t need to rely on env vars being correctly set by an invoking shell (json to rescue).
The cfg object instance becomes a parameter to class constructors or module functions.
If the idea of passing around a cfg instance is appalling, then hello global / singleton pattern!
Depending on what config data you need it might be a good idea to use environment variables. If all you need are server locations and credentials then environment variables are likely your best bet.
If you need fancy JSON or something else, global variables are nice.
It’s always interesting to read about best practices that advise against using a certain feature of a platform. If that feature is so bad, why was it added as a feature in the first place?
If the feature best fits your use case, use it with the understanding of the reasons the best practice advises against it and decide for yourself if you can accept and manage those concerns.
- Parse the config in main
- create and wire your objects passing any necessary values
- ???
Why every class creates its own logger? That might make sense, but looks suspicious
Use global imported variables for configs.
