[Twisted-Python] IUnicodeUsernamePassword?
Hi, Adding t.conch's ssh manhole (any third party IUsernamePassword provider would've done, really) to my app recently uncovered some unicode handling issues and I'm wondering what the appropriate way to resolve them is. In my API (AMP) there is a: - e-mail which is unicode - password which is unicode In my store (Axiom), there is a: - uid (unique user identifier, think UUID) which is bytes - e-mail which is unicode - password which is bytes, because it's the output of scrypt. scrypt also takes bytes as input: the password is first utf-8 encoded. Previously I was mistakenly handing around IUsernamePasswords where the e-mail address was unicode. Obviously the ICredentialsChecker implementation should take IUsernamePasswords that actually satisfy their contract and don't just pretend to, so ALL credentials it receives should have str/str, no exceptions. So, right now my credentials checker decodes the provided username to unicode in order to find the user by e-mail address (since, as mentioned above, Axiom thinks my e-mail addresses are unicode). The flip side of this is that the login method now first encodes the provided username and password as unicode. This is at least a semantically correct solution, but has the somewhat surprising property that in the 99.9% use case, the e-mail is received as unicode, immediately encoded, and then virtually straight after decoded again. That doesn't mean it's wrong, but it's a little weird. Potential solutions: 1. Make e-mail addresses bytes always. I don't know if this is semantically valid. 2. Add an IUnicodeUsernamePassword interface and implementation, and leaving adaptation to handle decoding/encoding. The downside here is that IIUC cred does not try to adapt to the ICredentialsChecker's supported interfaces, something I believe I complained about a few years back. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] twisted server config: command line, environment or Axiom?
Hi,
I have a bunch of configuration for my service. Things like endpoints, location
of the root axiom store… Right now I grab these from the environment. That
means that my twistd IServiceMaker plugin's options is pretty much by
definition empty.
I'm wondering if more mature twistd users have already learned the hard way
what the best way to manage this is. I see three possible solutions:
1. environment variables as I use them now: portable, simple.
2. command line options: still pretty portable and simple. additional benefit:
easy discoverability of options
3. axiom store: has the benefit that there's a single source of truth for
everything, i.e. I copy a store and I can recreate the service based on that
store exactly, using just the store
Like I said, I currently use (1), it looks like (1) and (2) are more or less
equivalent, I'm not sure if (3) is a good idea. I'm not really unhappy with
(1), it's just that finally writing my ServiceMaker class made me contemplate
the options.
(3) has the downside that it conflates user data with configuration data. I'm
not sure that's a good thing or not. It sounds very similar to axiomatic, which
I've heard of, seen, but never played with.
I already use environment variables to decide whether or not to run certain
tests ("live" functional high level tests, with real disk and network IO, that
take way longer than the unit tests to run). I'm going to keep it that way,
because there's no obvious way of passing configuration to the tests other than
the environment.
cheers
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted server config: command line, environment or Axiom?
On Wed, Aug 08, 2012 at 03:58:27PM +0200, Laurens Van Houtven wrote:
> Hi,
>
> I have a bunch of configuration for my service. Things like endpoints,
> location of the root axiom store? Right now I grab these from the
> environment. That means that my twistd IServiceMaker plugin's options is
> pretty much by definition empty.
>
> I'm wondering if more mature twistd users have already learned the hard way
> what the best way to manage this is. I see three possible solutions:
>
> 1. environment variables as I use them now: portable, simple.
> 2. command line options: still pretty portable and simple. additional
> benefit: easy discoverability of options
Both, preferrably. Defining all of your configuration in terms of
t.p.usage.Options is not a bad thing at all.
It's human readable, and scriptable. And it maps directly to ENV vars or INI
files (etc). If you have strange configuration requirements, (like binary
files), I think those belong in their own seperate file(s). E.g. SSL
certificates, PW files, etc
> 3. axiom store: has the benefit that there's a single source of truth for
> everything, i.e. I copy a store and I can recreate the service based on that
> store exactly, using just the store
Yeah, I hate that kind of configuration. Configuration is/should-be diffable
with standard tools, if possible.
> Like I said, I currently use (1), it looks like (1) and (2) are more or less
> equivalent, I'm not sure if (3) is a good idea. I'm not really unhappy with
> (1), it's just that finally writing my ServiceMaker class made me contemplate
> the options.
>
> (3) has the downside that it conflates user data with configuration data. I'm
> not sure that's a good thing or not. It sounds very similar to axiomatic,
> which I've heard of, seen, but never played with.
Well as indicated *I don't like it*, but really that is only due to lack of
tooling support... :)
> I already use environment variables to decide whether or not to run certain
> tests ("live" functional high level tests, with real disk and network IO,
> that take way longer than the unit tests to run). I'm going to keep it that
> way, because there's no obvious way of passing configuration to the tests
> other than the environment.
Yeah. Plain ENV vars are probably the best bet there.
--
Cheers,
-E
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted server config: command line, environment or Axiom?
On Wed, Aug 8, 2012 at 8:58 AM, Laurens Van Houtven <_...@lvh.cc> wrote: > Hi, > > I have a bunch of configuration for my service. Things like endpoints, > location of the root axiom store… Right now I grab these from the > environment. That means that my twistd IServiceMaker plugin's options is > pretty much by definition empty. > > I'm wondering if more mature twistd users have already learned the hard > way what the best way to manage this is. I see three possible solutions: > > 1. environment variables as I use them now: portable, simple. > 2. command line options: still pretty portable and simple. additional > benefit: easy discoverability of options > 3. axiom store: has the benefit that there's a single source of truth for > everything, i.e. I copy a store and I can recreate the service based on > that store exactly, using just the store > > Never, ever store config data and user data in the same place. Likewise, if you're going to persist config data, it should be in some kind of plain text format. 1) and 2) are fine, though I prefer a simple text file format as well, and I like to have multiple levels of configuration which are overridable. e.g. one or more config files (maybe in /etc, in /home/, the pwd) which override each other in a specified manner, which are in turn overridable by command line options and/or env vars whether any of this is appropriate for your case, and how much complexity you need obviously depends on your application Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted server config: command line, environment or Axiom?
On 01:58 pm, _...@lvh.cc wrote:
>Hi,
>
>I have a bunch of configuration for my service. Things like endpoints,
>location of the root axiom store… Right now I grab these from the
>environment. That means that my twistd IServiceMaker plugin's options
>is pretty much by definition empty.
>
>I'm wondering if more mature twistd users have already learned the hard
>way what the best way to manage this is. I see three possible
>solutions:
>
>1. environment variables as I use them now: portable, simple.
>2. command line options: still pretty portable and simple. additional
>benefit: easy discoverability of options
>3. axiom store: has the benefit that there's a single source of truth
>for everything, i.e. I copy a store and I can recreate the service
>based on that store exactly, using just the store
This doesn't really seem like a Twisted question... but...
axiom is my favorite configuration persistence tool.
>Like I said, I currently use (1), it looks like (1) and (2) are more or
>less equivalent, I'm not sure if (3) is a good idea. I'm not really
>unhappy with (1), it's just that finally writing my ServiceMaker class
>made me contemplate the options.
>
>(3) has the downside that it conflates user data with configuration
>data. I'm not sure that's a good thing or not. It sounds very similar
>to axiomatic, which I've heard of, seen, but never played with.
It doesn't conflate user data with configuration data any more than any
other system. Keep a database for your configuration and keep your user
data somewhere else (perhaps in a separate axiom store).
Jean-Paul
>I already use environment variables to decide whether or not to run
>certain tests ("live" functional high level tests, with real disk and
>network IO, that take way longer than the unit tests to run). I'm going
>to keep it that way, because there's no obvious way of passing
>configuration to the tests other than the environment.
>
>cheers
>lvh
>
>
>
>
>___
>Twisted-Python mailing list
>Twisted-Python@twistedmatrix.com
>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted server config: command line, environment or Axiom?
On Wed, Aug 8, 2012 at 3:58 PM, Laurens Van Houtven <_...@lvh.cc> wrote: > (3) has the downside that it conflates user data with configuration data. I'm > not sure that's a good thing or not. It sounds very similar to axiomatic, > which I've heard of, seen, but never played with. As exarkun mentions in his reply, you don't have to store the configuration and user data in the same Axiom store. If you're interested, the way "axiomatic start" works is that it locates all IService powerups on the store you pass it (which would be the "site store", in Mantissa), and attaches those to the application. I think most of the interesting IService implementations are in Mantissa, not Axiom, but axiomatic itself does not depend on Mantissa and of course there's no reason you can't write your own IService powerups. -- mithrandi, i Ainil en-Balandor, a faer Ambar ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] IUnicodeUsernamePassword?
Le Aug 8, 2012 à 5:36 AM, Laurens Van Houtven <_...@lvh.cc> a écrit : > 2. Add an IUnicodeUsernamePassword interface and implementation, and leaving > adaptation to handle decoding/encoding. The downside here is that IIUC cred > does not try to adapt to the ICredentialsChecker's supported interfaces, > something I believe I complained about a few years back. And it never will. Trying to make this work all the way would require infinitely transitive adaptation, not to mention some kind of additional plugin interface to make sure that all your adapter registrations get loaded by the right moment, so you need to have checkers which explicitly declare their support. But nowadays we have strcred, and if only everything used that, you could develop all this support in your application, including a strcred prefix which could fix any IUsernamePasword-checking checker by wrapping it :). -g___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
