Best practice for config files?
I recently wrote a program to do some record-keeping for me. I found myself hard-coding a bunch of different values into it. This didn't seem right, so I made my first use of configparser.ConfigParser(). Created the configuration file and everything is working fine. However, I wrote it based on the assumption that the program is running in the directory where the config file is stored, and has a specific name. I started having some second thoughts here. I thought about putting the location of the configuration file in the configuration file, but that seemed like a non-starter.[1] Should I specify the location of the config file with a command-line option, or is requiring the program to be executed in the directory containing the configuration file considered acceptable practice? [1] See Tegan Jovanka in _Castrovalva_ for more on this idea. -- Michael F. Stemper If it isn't running programs and it isn't fusing atoms, it's just bending space. -- https://mail.python.org/mailman3//lists/python-list.python.org
Re: Best practice for config files?
On 5/22/2025 7:09 PM, Rob Cliffe via Python-list wrote: On 22/05/2025 23:45, Mats Wichmann wrote: On 5/22/25 13:59, Michael F. Stemper via Python-list wrote: I recently wrote a program to do some record-keeping for me. I found myself hard-coding a bunch of different values into it. This didn't seem right, so I made my first use of configparser.ConfigParser(). Created the configuration file and everything is working fine. However, I wrote it based on the assumption that the program is running in the directory where the config file is stored, and has a specific name. I started having some second thoughts here. I thought about putting the location of the configuration file in the configuration file, but that seemed like a non-starter.[1] Should I specify the location of the config file with a command-line option, or is requiring the program to be executed in the directory containing the configuration file considered acceptable practice? Eh, there are so many context-dependent practices that it's largely impossible to identify something to call a "best practice" these days. I'm going to make a couple of comments, but there's more to consider than just these thoughts. There's a sort-of-standard package called platformdirs (a successor to appdirs) which tries to tell you where to put things in a platform- independent way. https://pypi.org/project/platformdirs/ https://pypi.org/project/appdirs/ This works nicely if you're building something to distribute, and you want to have a system-neutral way to access a config file. For personal stuff, that may be overkill. If it's only for you, just do what works. Enhancement: have a default location for the config file hard-coded, but provide a cmdline option to give an alternative. Mats is absolutely correct that there is no "one size fits all" solution. However one solution that might or might not suit you (at least on WIndows; I hope it's applicable to Unix) is to look at sys.path and search for your configuration file in every directory it lists until you find it (or give an error message if it isn't). Alternatively look at the PATH envronment variable, which contains a list of directories separated by semicolons and which you can access as os.environ['PATH'] . Best wishes, Rob Cliffe There is really no reason for a config file to be on the system path. On Windows, if it's not in the program's home directory (or a subdir of it), then it's common to be in the user's home directory. That's assuming the program is expected to have possibly different configurations for different users. Otherwise there's a \Users\Public directory (or C:\Users\Public\Documents) that could be used, if all users are desired to use the same config file. Or, again for all users, in c:\ProgramData, as long is you don't mind having to write the config file with admin privileges. The most flexible plan would be to have the program look in all those likely places for its config file. For more flexibility - but this is unnecessarily complicated for many applications - have the program look first on the command line for the config directory, then for an environmental variable, then in those likely places. -- https://mail.python.org/mailman3//lists/python-list.python.org
Searching for a file
It occurs to me that it might be useful if Python provided a function to
search for a file with a given name in various directories (much as the
import.import_lib function searches for a module in the directories in
sys.path).
This function would perhaps be best placed in the os.path or os modules.
To start the ball rolling, I offer this version:
import sys
import os
def findfile(fileName, *PathLists):
if not PathLists:
# Default to searching just sys.path:
PathLists = [ sys.path ]
# This may or may not be the most useful default;
# there is a case for having no default, forcing calls of this
function
# to explicitly specifiy which paths they want to search,
# and perhaps raise an Exception if they don't;
# suggestions welcome.
for pList in PathLists:
# if pList is a str, assume it is a semicolon-separated list of
directories
# (like the PATH and PYTHONPATH environment variables);
# otherwise assume it is a sequence of paths (like sys.path):
if isinstance(pList, str):
pList = pList.split(';')
for path in pList:
filePath = os.path.join(path, fileName)
if os.path.isfile(filePath):
return filePath
# Return first match found.
# Another possibility is to return a list of all matches.
return None
Then e.g.
print(findfile('os.py'))
would print something like
C:\Python311\Lib\os.py
This idea was partly inspired by Michael Stemper asking about best
practice for placing, and finding, a configuration file.
Thoughts?
Best wishes
Rob Cliffe
--
https://mail.python.org/mailman3//lists/python-list.python.org
Re: Best practice for config files?
On 5/22/25 13:59, Michael F. Stemper via Python-list wrote: I recently wrote a program to do some record-keeping for me. I found myself hard-coding a bunch of different values into it. This didn't seem right, so I made my first use of configparser.ConfigParser(). Created the configuration file and everything is working fine. However, I wrote it based on the assumption that the program is running in the directory where the config file is stored, and has a specific name. I started having some second thoughts here. I thought about putting the location of the configuration file in the configuration file, but that seemed like a non-starter.[1] Should I specify the location of the config file with a command-line option, or is requiring the program to be executed in the directory containing the configuration file considered acceptable practice? Eh, there are so many context-dependent practices that it's largely impossible to identify something to call a "best practice" these days. I'm going to make a couple of comments, but there's more to consider than just these thoughts. There's a sort-of-standard package called platformdirs (a successor to appdirs) which tries to tell you where to put things in a platform-independent way. https://pypi.org/project/platformdirs/ https://pypi.org/project/appdirs/ This works nicely if you're building something to distribute, and you want to have a system-neutral way to access a config file. For personal stuff, that may be overkill. If it's only for you, just do what works. Enhancement: have a default location for the config file hard-coded, but provide a cmdline option to give an alternative. -- https://mail.python.org/mailman3//lists/python-list.python.org
Re: Best practice for config files?
On 22/05/2025 23:45, Mats Wichmann wrote: On 5/22/25 13:59, Michael F. Stemper via Python-list wrote: I recently wrote a program to do some record-keeping for me. I found myself hard-coding a bunch of different values into it. This didn't seem right, so I made my first use of configparser.ConfigParser(). Created the configuration file and everything is working fine. However, I wrote it based on the assumption that the program is running in the directory where the config file is stored, and has a specific name. I started having some second thoughts here. I thought about putting the location of the configuration file in the configuration file, but that seemed like a non-starter.[1] Should I specify the location of the config file with a command-line option, or is requiring the program to be executed in the directory containing the configuration file considered acceptable practice? Eh, there are so many context-dependent practices that it's largely impossible to identify something to call a "best practice" these days. I'm going to make a couple of comments, but there's more to consider than just these thoughts. There's a sort-of-standard package called platformdirs (a successor to appdirs) which tries to tell you where to put things in a platform-independent way. https://pypi.org/project/platformdirs/ https://pypi.org/project/appdirs/ This works nicely if you're building something to distribute, and you want to have a system-neutral way to access a config file. For personal stuff, that may be overkill. If it's only for you, just do what works. Enhancement: have a default location for the config file hard-coded, but provide a cmdline option to give an alternative. Mats is absolutely correct that there is no "one size fits all" solution. However one solution that might or might not suit you (at least on WIndows; I hope it's applicable to Unix) is to look at sys.path and search for your configuration file in every directory it lists until you find it (or give an error message if it isn't). Alternatively look at the PATH envronment variable, which contains a list of directories separated by semicolons and which you can access as os.environ['PATH'] . Best wishes, Rob Cliffe -- https://mail.python.org/mailman3//lists/python-list.python.org
Re: Best practice for config files?
On 2025-05-23, Thomas Passin wrote: >> Alternatively look at the PATH envronment variable, which >> contains a list of directories separated by semicolons and which you can >> access as os.environ['PATH'] . > > There is really no reason for a config file to be on the system > path. On Unix/Linux I would _never_ expect a config file to be in the PATH. PATH is for executables. Executables and config files are not stored together. That's simply not how it's done on Unix. There is sort of a traditional set of locations where applications look to find a config file: $HOME/ $HOME/.config/ $HOME/.config// /etc/ /etc// /usr/local/etc/ /usr/local/etc// The last two overried all of the others. Config files that reside in $HOME/ usually start with a dot. Often they end in 'rc'. Config files in other directories usually don't start with a dot. There's usually an directory only when an app needs multiple config files. If an app only has one config file, tradition is that you don't need a directory for it. Many applications will parse two config files: a global one from /etc or /usr/local/ and a user-one from somewhere under $HOME. However, it varies a lot from one application to another... > On Windows, if it's not in the program's home directory (or a subdir > of it), then it's common to be in the user's home directory. That's > assuming the program is expected to have possibly different > configurations for different users. > > Otherwise there's a \Users\Public directory (or [...] > > The most flexible plan would be to have the program look in all > those likely places for its config file. For more flexibility - but > this is unnecessarily complicated for many applications - have the > program look first on the command line for the config directory, > then for an environmental variable, then in those likely places. -- https://mail.python.org/mailman3//lists/python-list.python.org
