OrderedDictionary

I have often wished to create a keyed collection of objects (i.e. a Dictionary) in which the elements could be accessed by key or iterated over in the order in which the elements were added (i.e. treat the collection as either a Dictionary or an OrderedCollection.)

I have seen a few implementations of dictionary classes that retain the insertion order of their elements, but they all seemed to be overly complicated to me. I created this OrderedDictionary by overriding just 5 methods of Dictionary. It remembers the insertion order by maintaining a parallel ordered collection of the associations in an instance variable.

Instances of OrderedDictionary behave as Dictionaries in all respects except that when associations, keys, or values are iterated over, the order in which the iteration occurs is the same as the order in which the associations were originally added to the OrderedDictionary instance.

OrderedDictionary is in Squeaksource

Coding for the Man vs Coding for Myself

Ok. I spent over 20 years programming in corporations. For the last year or so I have been programming for my own enjoyment creating web apps and iPad apps. How is my programming behavior when I'm programming for myself different from when I'm programming for my employer? How does it feel?

Well, working for myself, I don't often take on a project that looks boring or that is simply a repeat of something I have done before. Working for a productivity-enthralled corporation, I often worked on things I had done before or spent time guiding others in doing things I had done before because that is more "productive". (Productive being defined as whatever produces the largest payout for the corporate officers. I used to think it was whatever benefited the shareholders most, but that has proven not in fact to be the case.)

Doing something over even once when working for a corporation is seen as an admission of failure or incompetence, so a large amount of effort is spent in design to make sure that it is done right the first time. Of course it is not always done right the first time, so additional effort is spent trying to avoid simply admitting failure and starting over -- which would have been the right thing to do in many cases.

I really only write applications for myself. If I wouldn't be interested in using it, then I'm probably not qualified to write it. Contrast that with the corporate world where many programmers don't even understand what the application they're working on is for.

When I code for myself I usually start in what amounts to a prototyping mode -- just make it work; don't worry about architecture, memory leaks, patterns or anything. Just make the application work and demonstrate that all the essential algorithms and the UI flow are understood and uncover any hidden gotchas. I usually uncover the possibility of great features that I didn't originally think of and discover that some features I thought were really cool are not in fact very useful and seem to be difficult to implement properly in any case.

So the definition of the product is influenced by the experience of building it and trying to use it in the first phase of development. In a corporate shop, that level of feedback is generally non-existent -- the barrier between the programmers and the domain experts for the product is so high that product specifications are essentially given -- and the programmers couldn't judge whether a feature is useful for the end user or not anyway.

Then I re-factor the prototype to create the final product, copy-and-pasting paragraphs of code into new classes and often re-writing the same piece of functionality many times until I either can't think of how to make it better or until I'm actually satisfied. At this stage, there are times when the code is so broken that it won't even run. Imagine this in a corporate shop: 'What do you mean you're making it better? It worked yesterday and it doesn't work now. Shouldn't you be spending your time fixing something that is broken instead of breaking something that works?"

I don't have a schedule. It's done when it's done and its done when I'm not embarrassed to release it. I may decide that a feature is not worth the bother and remove it -- and wait to see if someone requests it after the app is deployed -- but not because I'm being measured on meeting some arbitrary schedule.

I never start a project if I already know how to do everything required. That would be boring. I want to discover new technologies, alternatives to the way I have been doing things, and learn new stuff. I could write another C# Windows program using Visual Studio on a PC -- or I could write an iPad application in Objective-C using Xcode on a Mac -- or I could write a web-based app using Pharo Smalltalk and Seaside on Linux. It's just more interesting to learn the alternative platform tools than to keep doing the same thing over and over. And I find that I can make better judgements about the quality of the platforms involved than I could when I relied on prejudice and ignorance.

I heard somewhere that an expert is someone who learns more and more about less and less until he knows all there is to know about nothing at all...

TFFiler: Managing external files via Monticello

My problem was that I had a number of external files that were required for my Seaside web application. These were CSS, Javascript, and image files that had to be placed in a specific directory tree structure. When I moved my web app to another system, I always had to remember to reconstruct the external directory tree. Over time, the directory tree got larger and more complex.

For deployment, I wanted the files to be served by Apache and not the Seaside ExternalFileLibrary. (ExternalFileLibrary was the most convenient approach during development.) This meant the directory tree should be moved to a location other than the Pharo Resources directory, which was most convenient during development.

The current version of Monticello does not provide a way to save external files with a project. TFFiler addresses this shortcoming.

TFFiler

TFFiler creates file container classes that can contain the content of many files and can be saved to and loaded from Monticello. These file container classes can restore their files on demand or on first load.

Here's an example. Imagine you have a directory called static in the Pharo Resources directory that contains subdirectories containing CSS, Javascript, and image files. Here's how you would create a StaticFiles class containing that directory tree:

 TFFiler loadDirectory: 'static' into: 'StaticFiles'.

The StaticFiles class will be created. It will contain methods to

• List the files
• Display file content in a workspace
• Add additional files
• Remove files
• Replace files
• Restore the static directory tree
• Restore individual files

Here's how StaticFiles is used to re-create the static directory tree in the Pharo Resources directory:

 StaticFiles restoreDirectory.

If you wanted the static directory to be created when StaticFiles was first loaded from Monticello, you would place the following in the class initialize method:

 self restoreDirectory

If you wanted to restore the files to a directory other than static in Pharo Resources, you would use something like this:

 StaticFiles restoreFilesTo: '/var/www/mywebapp/assets'.

The StaticFiles class comment contains a list of all the files contained in the class.

Note that files are identified by their restorePaths rather than filename. The restore path may contain directories that will be created if needed when the file is restored. The restore paths do not include the directory from which the file was loaded. For example, a CSS file in a subdirectory of the static directory might have the restore path css/mywebapp.css.

API

The public API of TFFiler is in categories load, load-not-copied, and restore. The load-not-copied methods are not copied into the created file container class while all the other methods are by default. Note that you may create file containers in any package and you may exclude load methods from file containers if you wish.

categories load and load-not-copied

1. loadDirectory:
2. loadFile:restorePath:
3. removeFile:
4. removeFile:From:
5. loadDirectory:into:
6. loadDirectory:into:packageName:
7. loadDirectory:into:packageName:restoreOnly:
8. loadDirectory:into:restoreOnly:
9. loadFile:into:restorePath:
10. loadFile:into:restorePath:restoreOnly:

category restore

1. displayFile:
2. restoreFile:to:
3. restoreFilesTo:
4. restorePaths

Internals

TFFiler-created classes store the file data in base-64-encoded compressed strings answered from class methods named with the base-36-encoded file path. A private #fileList class method is created that returns a dictionary of the file data methods indexed by their restore paths. This method is used, for example, by #restorePaths to answer a sorted collection of the restore paths (files) contained in the container class.

All files are read and written in binary format.

TFFiler has been tested on Linux, OSX, and Windows. Files can be restored on a platform different from the one on which they were acquired.

TFFiler is available from Squeaksource

Consolidated Authentication - why its a bad thing

This is a reprint of an email I wrote a while ago. See in in context here: authentication for seaside.

Consolidated authentication in this article refers to the practice of using a single authentication provider, like Facebook, Twitter, Yahoo!, or Google to login to multiple sites. Convenient but dangerous.

Regarding consolidation of account login security:

I think it depends on the purpose of the password and the account. If the account exists only to separate one user's data from another's, then one could argue the password is actually not needed at all; the username is enough. If, in contrast, the purpose of the password is for security, then the password is a critically important part of preventing unauthorized access to the user's information.

Users have for years been using the word "password" and other easy-to-guess words as their password and many of these users have suffered the consequences. Entrusting the security of all your on-line accounts to a single entity, be it Facebook, Twitter, or a national government provides a single point of failure for the security of the associated accounts. This is the same reason why using the same password for multiple accounts is ill-advised.

Passwords are vulnerable not only to on-line hacking, but also to theft or hacking from within the organization that maintains and verifies the password. I believe the threat from inside the password-holding organization is probably as great or greater than the threat from outside given the greater level of access those inside the organization have.

I have divided my on-line accounts into two groups: those whose security is not important because they do not contain any personal information, and those whose security is indeed important, such as on-line bank accounts and any account containing personal information that could lead to identity theft. I use one password for all the insecure accounts, and a different password for each of the secure accounts. That way if a password is revealed, only one account is immediately compromised.

I understand keeping track of many passwords is inconvenient and just automatically using one's Facebook login at another site is very convenient. Convenience is also the reason why people use the word "password" as their password. I, personally, would not use automatic Facebook or Twitter login for any but my insecure accounts -- and those are almost by definition, the accounts that are not very important to me.

I have three friends whose on-line accounts were compromised and who lost significant amounts of money and suffered months of continued problems recovering from identity theft. These were not rich people. This does happen.

I think there is still a place for per-site login and security, inconvenient as it may be.

See this Coding Horror article: youre-probably-storing-passwords-incorrectly.html.

I use Husharoo to store my secret stuff securely. Husharoo doesn't keep a copy of my cipherkey at all, even in encrypted form and the encryption and decryption is done on my computer, not on the server.

Seaside session stealing

One session at a time, please

I was recently working on a web app in which it was necessary to ensure that only one session at a time was active per user. The app in question is highly interactive and accumulates information from the user but does not commit it to the data store more often the once every few minutes (or on session termination) in order to limit disk write frequency.

If the user should log in a second time from another browser window or computer without first logging out their old session, I needed to move the accumulated information to the new session and invalidate the old session.

This proved to be a little tricky and I gained some insight into Seaside session handling in the process.

Maybe this shouldn't really be called session stealing, but rather session data transfer or something, but stealing sessions sounds like so much more fun.

Some background

The application in question uses TFLogin for managing logins and for saving per-user data. However, the general approach should be usable with other user management schemes.

Two things:

1. The TFLogin User object contains an applicationProperties dictionary into which the application may place per-user objects and have them persisted along with the rest of the User object when TLLoginComponent>>#saveUser or TLLogincomponent>>#logout is sent.
2. The block provided to TLLoginComponent>>#onAnswer: is evaluated after successful logins in the user's session. This is where the skulduggery begins... First we send #logoutOldSessions to self.

#logoutOldSessions

This method is called when the user logs in to logout any old session of the user and steal its user data objects.

We look through all the sessions for our application. Ignoring sessions with no user (these are sessions not yet logged in), other user's sessions, and our own session, we are left with - at most one - session which is a previous session of the current user. (At most one, because this method is executed on every login and logs out any previous session that it finds.)

We send #logout: to the previous session's presenter instance, passing the session to be logged out. We pass the old session because self>>#session will always return the current session, no matter which application instance's code is being executed.

(#logout is described in more detail in the next section.)

Here is the code for #logoutOldSessions that is called from within the TLLoginComponent>>#onAnswer: block:

logoutOldSessions
  "Find any old session that hasn't saved its user data, retrieve
  the user data from it and log it out. There should be at
  most one such session."
	
  | otherSessionUserData |
  self session application sessions do: [ :each |
    each user isNil ifFalse: [
      each user userId = self session user userId ifTrue: [
        each == self session ifFalse: [
          otherSessionUserData := each presenter logout: each ]]]].
  otherSessionUserData isNil 
    ifFalse: [
      self session user applicationProperties
        at: 'userdata'
        put: otherSessionUserData].

#logout:

This method is meant to be invoked from a different session than the one in which the app was instantiated. The session is passed as the argument to the method.

First we save our possibly uncommitted user data in a temporary variable. The we send #logout: to our TLLoginComponent instance, passing the session we were provided. (TLLogincomponent>>#logout: is a special alternative to the plain TLLogincomponent>>#logout method. It is provided precisely to allow logout of one session by another session. This special method does not save the user object but simply sets it to nil.)

We then abort the background process that is in charge of periodically saving modified user data and unregister the session.

Finally, we return the userdata to the calling #logoutOldSessions method, where it will be used to initialize the new session.

logout: session
  "This is invoked by other sessions."
  | userdata |
  userdata := session user
    applicationproperties at: 'userdata'.
  loginComponent logout: session.
  session abortBackgroundProcesses.
  session unregister.
  ^ userdata

Conclusion

This has proven to be reliable and avoids using the disk to transfer data between sessions unnecessarily. There remains what to do when the user returns to the old session, which still appears to be logged in from the browser's viewpoint, and tries to continue work there. There is a solution to this, but that is a subject for the next article...