UniConf consists of many collaborating objects. The newly refactored UniConf differs from the original primarily in that each object now embodies as few behaviours as possible. Delegation is used throughout the architecture to ensure that no one object acquires too much responsibility. The Abstraction?s work together to provide Cohesion and Closure while DataHiding? ensures that Coupling is minimized.
This page is dedicated to recognizing the (sometimes non-obvious) forces that are involved in designing an abstraction such as UniConf. I hope that greater understanding of these forces will help implementors continue to improve the new UniConf while avoiding some of the traps of its predecessors.
Since I expect there to be a lot of text here, I would prefer if comments were added to the bottom of the page rather than interspersed within the body.
--jbrown
The Original UniConf
The original implementation was characterized primarily by three parts, UniConf, UniConfGen, and UniConfEvents?. UniConfGen defined an interface for generators that could load and store keys from a variety of data sources. It depended on UniConf to expose these keys, to store their values locally, maintain caching information, and provide tree structure. UniConfEvents? was a set of classes that manipulated flags managed by UniConf to provide event notification. So UniConfGen and UniConfEvents? both delegated many of their responsibilities to the a single catch-all UniConf class. Eventually, it provided the following facilities:
- primary client-side interface
- handle for a subtree
- tree structure
- key lookup
- local key and value backing storage
- value inheritance
- caching
- flag storage for notification support
- generator mountpoint management
Futhermore many clients of the UniConf class were directly responsible for:
- instantiating and installing the correct generator and supplying it with a reference to the UniConf tree and any streamlists it needed (lack of generator transparency)
- remembering to unlink the inherited default tree
- checking and maintaining dirty / cache / notification flags
- forming, parsing, and normalizing key names as strings (UniConfKey? was not used pervasively)
Some limitations of that design were evident in the answers to the following questions:
- When do UniConf branches get destroyed?
- Who resolves conflicts if generators provide the same keys?
- Which generator owns which keys?
- How can UniConf be modified to support multiple levels of inheritance?
- How can UniConf be modified to support multiple generators at a single mount point?
- How does a generator keep the local cache in sync?
- What can be done about resource consumption if a generator already has a database in memory that it wants to publish through the UniConf interface? (How to avoid copying the entire database?)
- How can a generator provide incremental population of the UniConf tree? (How to support lazy loading?)
- How can a generator be made read only?
- How can UniConf be extended to enforce access permissions?
Most of these problems stemmed from a lack of Closure in the design, and from inadequate DataHiding? or insufficient Abstraction?. Unfortunately, the older UniConf shoehorned itself into a box that was two sizes too small which required some heavy refactoring to remedy.
The Grandparent, WvConf?
It is interesting to read over the WvConf? API. At its core, it is simple, purpose-built, and efficient enough for the problems it was designed to solve. Though it is non-hierarchical nature and its imposes some constraints on the type and quantity of data it can effectively manage, it fulfills its requirements surprisingly well. Where it fails is mostly in the details. Some convenience has been bought at the expense of confusion or loss of generality.
Some problems:
- The nomenclature in the interface is cluttered. There is confusion over when underscores should be used, and when words should be concatenated. There is the somewhat unfortunate choice of "setbool" in the "add_setbool" and "del_setbool" family when considered alongside the "set", "setint", "get", "getint" family. Functions that search several sections at a time are prefixed with "fuzzy" which is not sufficiently meaningful.
- The "fuzzy" functions are only used by the WvDial program, so it is unclear whether they are worth the expense of including them in the interface. Moreover, they are difficult to use since they accept a WvStringList? as first argument.
- The salient feature of "setraw" and "getraw" is that they parse fully-qualified keys. The "raw" part has nothing to do with the type or manner in which the data is returned as is suggested when compared with "get", "getint", "set", "setint". Possibly a better choice would have been to expose the key parsing more effectively.
- The "maybeset" function sets a key if it is unset. The choice of "maybe" leads one to ask, when it will actually set a key (once every blue moon?). A more effective choice might have been "setifunset", or perhaps just a boolean flag added to the "set" function argument list with a default value of "set always".
- The "passwd" functions do not belong at all in WvConf?, they belong in Weaver. The comment "not defined here" says it all. The keychain (password repository) responsibility is better left to some other smarter object that knows how to serialize its controlled data to a ubiquitous string, or possibly directly to WvConf? (though then it could not be used with UniConf, aha!).
- The "addname" and "setbool" callback features might be more effectively represented if WvStreams callbacks were instead first-class observer objects. This would at once eliminate the need for the
void*pointer and provide a neat way of providing pre-cooked (convenience) callback types for a variety of purposes without bloating interfaces. Object identity is also a more meaningful and accessible relationship than callback,void*, section name, and key name equality. (A more general WvStreams design critique.) - The implementation contains a bit of duplicated code.
- A few internal functions have not been marked private.
Some Philosophy
The main theme at NITI is BuildForToday?. This works fine with in-house products such as the Weaver? but is not very suitable to the construction of class libraries like WvStreams. What seemed like a harmless shortcut yesterday, may constrain future growth tomorrow or make more difficult the maintenance cycle by obscuring interactions between objects. For example, rampant code duplication is often a sign that somewhere a year or two ago, someone should have spent an extra 5 minutes moving the shared code into a function or splitting it up, and instead duplicated it once or twice. This would be followed up by others until eventually the design erodes away to dust. For my WvStreams and UniConf improvements, I have instead chosen BuildForTodayPlanForTomorrow?. Closure is your friend. DataHiding? saves lots of headaches during all cycles of development, testing, debugging, deployment (for class libraries) and maintenance. Abstraction? is a double-edged sword which must be used wisely to promote evolution while still adhering to BuildForTodayPlanForTomorrow? since it is too easy to get carried away. However, beware what interfaces you supply your clients lest they turn them against you. Don't commit to contracts (semantics) that you cannot guarantee or that are too complex to maintain.
Architecture
Mounting Semantics
UniConf allows a tree of generators to be constructed much like the Unix VirtualFileSystem? does with its filesystems. In the common case for Unix, one filesystem is mounted at the root of the tree to provide the structure to which other filesystems attach. Unix permits any number of filesystems to be mounted at any mount point, provided that the mount point exists. UniConf relaxes the latter requirement such that mount points need not be physically created before generators are mounted. For example, in the case where generator A is mounted at path "/foo/bar/baz" and "/foo" exists, but not "/foo/bar" and consequently not "/foo/bar/baz", the existance of keys "/foo/bar" and "/foo/bar/baz" will be inferred to exist and have an empty value.
Scoping
Defn: Key in scope of generator
Let MP1 be a mount point path.
Let G1 be a generator mounted at MP1.
Let K be a key.
K is in the scope of G1 if MP1 is a prefix of K.
Defn: Most-specific generator
Let MP1 be a mount point path.
Let G1 be a generator mounted at MP1.
Let K be a key.
G1 is the unique most-specific generator with respect to K if all of the following conditions hold:
- K is in the scope of G1.
- If G2 is any other generator mounted at MP1, then G1 must have been mounted before G2.
- If MP2 is any other mount point such that MP1 is a prefix of MP2, then MP2 must not be a prefix of K, or MP2 must not have any mounted generators.
Defn: Next most-specific generator
Let K be a key.
Let G1 be the most-specific generator with respect to K.
G2 is the unique next most-specific generator with respect to K and G1 if unmounting G1 and all generators more specific than G1 causes G2 to become the most-specific generator with respect to K.
Defn: Least-specific generator
Let K be a key.
G2 is the unique least-specific generator with respect to K if there does not exist any generator G1 such that G2 is the next most-specific generator with respect to K and G1.
Defn: Degree of specificity
Let K be a key.
Let G be a generator.
The degree of specificity of G with respect to K is:
- 0 if G is the most-specific generator with respect to K.
- 1 if G is the next most-specific generator with respect to K and the most-specific generator with respect to K.
- n + 1 if G is the next most-specific generator with respect to K and G2 and G2 has a degree of specificity of n with respect to K.
- undefined if K is not in the scope of G.
Defn: Providing a key
Let MP be a mount point.
Let G be a generator mounted at MP.
Let K be a key.
G provides K if MP is a prefix of K, and the value in G of the key formed by stripping MP from the beginning of K is non-NULL.
Defn: Most specific provider of a key
Let G be a generator.
Let K be a key.
G is the unique most-specific provider of K if G has the smallest degree of specificity with respect to K among all generator that have K in scope and that provide K.
Defn: Key ownership
Let G be a generator.
Let K be a key.
G is the unique owner of K if G is the most-specific provider of K or if G is the most-specific generator with respect to K and no other generator provides K.
Integrity Constraints
- Key A has a non-NULL value if and only if it exists.
- Key A/B exists only if key A exists. Thus if key A/B exists, then key A must also exist.
FIXME: Need to talk about key Aliasing, Shadowing and all sort of complications there...
TODO: We the above has many implications on things like multi-mounts, defaults, filtering mounts, caching, etc...
Comments
jnc (2003/03/14): It is time to start thinking about how UniConf should report errors.
For a while it was possible to report a failure on setting or getting an individual key, which went away for two reasons: first was efficiency in the client protocol, and the second was that nobody ever checks them anyway. (If you're like me, you feel guilty if a return code exists but you don't check it, so I was very happy when they disappeared.)
I don't think it's a good idea to bring back this level of error reporting, but we do need a way to notice that a UniConfGen has completely failed. Its ini file has become scrambled, or its network connection has gone away, or its carrier pigeon has starved to death.
Obviously we want to give it a WvError? that we can check - the problem is when to check it. Unlike a WvStream?, uniconf objects aren't running in a handy select loop. We could simply check it on every access to the object, which means that if the pigeon dies while the app is off doing something else we won't notice until we want to actually use the pigeon. This is a little wasteful since it might take time to raise a replacement pigeon.
Also, we don't want to force the caller to call an isok() function after every operation, because they just won't.
I suggest adding a public callback to the UniConf and UniConfGen objects which will be called when an error occurs. It's up to the UniConfGen when to check its own error status if polling is required. (My first thought is to check on every set and get, and if it's convenient and not resource-intensive to check periodically when idle. UniClientConn?, for instance, has a WvStream? in the GlobalStreamList?, so as soon as that stream goes bad it can notice and pass the message on. No need to wait for the next access.)
At this point we can implement UniFilterGen?'s which handle common recovery strategies (such as UniFallbackGen? to try a list of backup generators).
It's tempting to declare that if a get or set fails (when called from a synchronous program) then the error callback will be called before the get or set returns. Or at least to make some sort of behaviour guarantee. Is this wise? Is this even helpful? It would be nice to know when the generator fails exactly how much of your valuable data was lost. But it's also tempting to declare that there are no guarantees at all, and probably more accurate.
Thoughts?
- apenwarr (2003/03/14): it's probably be safe to say this: if any UniConfGen can change from "no error" to "error" status asynchronously (ie. without actually calling a get() or set() or whatever to set it off) then it is a stream. If it is, then it should be responsible for noticing brokenness (like when a client is unceremoniously disconnected from its server) at a "reasonable" time.
I think every generator should have an isok(), and every UniConf handle should have an isok() that asks its generator whether it's okay. Maybe there should be a UniConfRoot?::is_all_ok() or something to check all the mounted generators for okayness. An error callback may or may not be handy - something tells me that, like exceptions, most people won't know what to do immediately when there's a problem, and we might want to just let them delay things until they're ready to deal with it.
I won't try to stop you if you want to add an error callback, though, if you're planning to use it immediately after you add it and it makes your code obviously better than just checking isok() occasionally.
- apenwarr (2003/03/14): Oh, by the way. Remember that anything attached to a "failable" generator should probably be wrapped in something non-failable (like a cache) anyhow, so even if your server goes away, gets and sets won't become complete nonsense. A generator should do its best to not return complete nonsense.
- ppatters (2003/03/14): Where part of this discussion started, is in the instance of, oh, WvPrint, WvDial and RetchMail, where once we uniconf them, then we'll want to have them do something like "If you have a UniConfDaemon running on the system, use it, if not, default to the ini file in /etc as normal". Thus it would be great if we could have a generator that takes a list of URI's ("tcp:localhost","ini:~/.wvdialrc", "ini:/etc/wvdial"), and have it mount as many of them as it can... currently, I don't think that you can do this... can you??
- jnc (2003/03/14): For this, it's probably enough to check isok() just after creation - except that it takes time to connect. (Right now we're hacking around this in the PhoneIntegrator? with a 1-sec pause between steps in startup - ewww.) I think for this a callback would be most appropriate because you do know exactly what to do when it fails - try the next generator - but not how long to wait before you do you the check.
- pphaneuf (2003/03/15): No, no, no... You can mount both, use the fast one, but set its default to the slow one. In one fell swoop, it allow allows you to override global defaults per host or something like that.
- ppatters (2003/03/14): Where part of this discussion started, is in the instance of, oh, WvPrint, WvDial and RetchMail, where once we uniconf them, then we'll want to have them do something like "If you have a UniConfDaemon running on the system, use it, if not, default to the ini file in /etc as normal". Thus it would be great if we could have a generator that takes a list of URI's ("tcp:localhost","ini:~/.wvdialrc", "ini:/etc/wvdial"), and have it mount as many of them as it can... currently, I don't think that you can do this... can you??
dgtaylor? (2003/04/16): Just a note: It is probably a very bad idea to have an assert on a moniker getting a generator... I'm leaving this line in right now, but it's quite annoying for certain things, say UniKonf? where if you don't have the moniker strings memorized perfectly you die.
- jnc (2003/04/17): That's a good philosophical question: are monikers meant to be user-visible? If not, UniKonf? should have a dropdown for "type of client" or something so that it can generate and verify the monikers in the code. (You can probably be fancy and get the complete list of monikers from the moniker registry or something - it's just a WvHash?, IIRC.) But that's probably more work than necessary ATM.
- apenwarr (2003/04/17): Well, I want to modify the moniker stuff to be queryable (so you can get a list of available monikers of a type, and their descriptions). That will let you have a self-configuring dropdown box. Even so, the assertion thing is pretty evil.