In the long run, we should be looking at these specific tools for the front end, but that can wait. Two archetectural concepts that have come up lately are more connected to how we should include these properties in the back end. At first I was thinking that making a clear split between the Card model and the history module was to allow us to skip all that work if we didn't need to refer to state before the current version. Now I'm thinking that the representation of the current state as the result of a flow of actions The cards and card_references tables can be re-generated if you have a store of the history. In fact, with the complete history, you can restore any state of the data, or compare them or ...
Remove activerecord dependency from Card. Instead implement a Store abstraction similar to the one used by Flux or Reflux, only instead of being a persistant store in a browser, it can use a storage model (active record to start). The key to this abstraction is that you never mutate a card directly, you send the update payload down the line in a change action to the Stores. Listener/observers to the actions and the resulting Store transformations signalled to downstream listeners trigger all the other changes and events that need to happen. Otherwise, the stores are 'get-only', you can only read data or push payloads at the store.
This would allow us to persistently cache objects. In a lot more places we would have to carry the equivalent of action_id for card_ids so that we don't have to break the cache when rules change. Any cached data whether in AR or a cache store would have the persistant and immutable properties. New data will have new action_ids to be cached or stored, but all these ids are internal to the store anyway. Memory and cache instances of an abstract card will be made concrete with an action_id (except when new?).
In case it isn't clear, this is more a thought experiment than anything at this point. It is interesting to think about where we use ids and where the need is to refer to any version vs. a specific version of a card. It is clear that the ids need to be assigned by the Store, and they are (by AR now). If it is necessary for card_ids and action_ids to be either 1) disjoint or 2) when there is a card_id and action_id that are the same (equal) the action is for that card. That might typically be the first action when the card is created, but it doesn't have to be.
WQL will be one of the harder parts to re-design in these terms, but I think the result could be very worthwhile. The parsing of queries probably doesn't change much, but the process of executing searches moves from being mostly a Card thing to being part of the Card Store. Anything related to the references table is also inside the Store (for support of queries), and similarly the cards table would be designed to support search by WQL as compiled into AR queries into Store tables.
It will also put the updating of references into the store. Card code won't need to know how references and history is represented in the Store, the Store just does what it needs to based on the actions in processes, and after that reads and searches will return the new data. The Store might be lazy about how much work it does at the point of commit, all it has to do is commit the actions to a transaction log (our history tables already are DB style roll-back/forward logs) and only update what's in the cache for a given card_id when the store is asked for the latest action on that card. The reference update queue only has be fully processes if you need to do a search now. You can be lazy about lots of stuff.
Immutability means that the Card objects have to obey the immutability contract. If you change it, it becomes a new object (which when stored/committed will get a new action_id), and mutators have to return new objects that you need to store locally (often overwriting a local variable that had the original one), but the more functional our code is, the more it just returns a new (vs. mutated) object up the function stack.
the needs section begins "Remove activerecord dependency from Card.". That's not a need, that's a step in an implementation plan. Can you connect this to needs?