From shriram@cs.rice.edu Tue Mar 16 01:00:42 1999 From: Shriram Krishnamurthi To: "Mitchell Wand" Cc: lieber@ccs.neu.edu, johan@ccs.neu.edu, dougo@ccs.neu.edu, lorenz@ccs.neu.edu, matthias@cs.rice.edu, mira@informatik.uni-siegen.de Subject: Adaptive Programming and functional combination Mitch, Thanks for your detailed comments. It seems we are speaking the same language, but we disagree on the likelihood of surprises. As you say, this is an empirical matter. But more on this below. (Re. order of arguments: I agree that it makes more sense to do things in the other order.) If the moral of your LC/fv example is that types do not solve the maintenance problem, I agree. (Heck, 311 students need no help from AP to write all sorts of bizarre, but type-correct, fv's.) BUT I am unconvinced that the LC example is characteristic of most OO extensions. Typical extensions seem to add fields and methods much more often than new classes of data. This is why the Interpreter pattern seems to be the standard way of laying out data, often as a starting point for more complex patterns. In other words, people subclass the variants more often than they add new variants. Since behavior cross-cuts variants, if you add new variants, you naturally have to think carefully about how these affect your operations! (Notice that your LC example is an instance of this. Indeed, a static analyzer can warn me when a traversal encounters a new variant.) But if you're adding new fields and methods, these typically don't affect the operations you were performing. Indeed, if you programmed in a "purely" adaptive fashion, your classes would contain only fields, no methods; then you would only be adding new fields, and doing little overriding. Karl talks about the "noise" of OO programs. I think the discussion above more precisely describes the maintenance-induced "noise". The other "noise" is there from the start. It comes about because OO methodologies perform perhaps overly detailed analyses of domains, and end up producing very elaborate class hierarchies to represent them. I do think that the types I described are useful even though they don't solve the maintenance problem. I think of an "otherwise" function as a combination of a "threader" (to use your term) and a "collector". The types are useful precisely because they suggest appropriate (and often natural) operations to use here. Threaders follow principally from inherited types, while collectors are induced primarily by synthesized types. (Eg: in the bus stop example, the synthesized type would be either a "set" or "list" of Person, depending on how we wanted to treat duplicates. The natural collectors would then be union and append, respectively.) If a traversal needs to switch types, I think the traversal should simply halt at that point, and a new traversal should take over. The type represents the coherent action being performed. The switch in types suggests a switch in action. I believe this is a design principle that will reduce surprises. Regards, 'shriram From wand@ccs.neu.edu Tue Mar 16 10:27:45 1999 To: shriram@cs.rice.edu Cc: "Mitchell Wand" , lieber@ccs.neu.edu, johan@ccs.neu.edu, dougo@ccs.neu.edu, lorenz@ccs.neu.edu, matthias@cs.rice.edu, mira@informatik.uni-siegen.de Subject: Adaptive Programming and functional combination >>>>> On Mon, 15 Mar 1999 23:59:49 -0600 (CST), Shriram Krishnamurthi >>>>> said: SK> Thanks for your detailed comments. It seems we are speaking the same SK> language, but we disagree on the likelihood of surprises. As you say, SK> this is an empirical matter. But more on this below. SK> ... I am unconvinced that the LC example is characteristic of most OO SK> extensions. Typical extensions seem to add fields and methods much more SK> often than new classes of data. This is why the Interpreter pattern seems SK> to be the standard way of laying out data, often as a starting point for SK> more complex patterns. SK> In other words, people subclass the variants more often than they add SK> new variants. Since behavior cross-cuts variants, if you add new SK> variants, you naturally have to think carefully about how these affect SK> your operations! (Notice that your LC example is an instance of this. SK> Indeed, a static analyzer can warn me when a traversal encounters a SK> new variant.) But if you're adding new fields and methods, these SK> typically don't affect the operations you were performing. Indeed, if SK> you programmed in a "purely" adaptive fashion, your classes would SK> contain only fields, no methods; then you would only be adding new SK> fields, and doing little overriding. That is an interesting insight. I don't have a lot of experience writing (and modifying!) sizeable OO programs, so my observations are based on the kinds of problems I have worked on. I am inclined to believe that adding methods would be unlikely to cause traversals/visitors to give wrong answers. Adding fields is somewhat different. Let's imagine adding a field whose "real" semantics doesn't change the result of some traversal. I think my&Johan's traversals would not be affected. But in the Demeter setup, adding such a field could easily change the result if the new paths to the target class were created. Of course, as Karl points out, one can imagine a static analysis that detects this. SK> Karl talks about the "noise" of OO programs. I think the discussion SK> above more precisely describes the maintenance-induced "noise". The SK> other "noise" is there from the start. It comes about because OO SK> methodologies perform perhaps overly detailed analyses of domains, and SK> end up producing very elaborate class hierarchies to represent them. Another factor that should be considered is that OO models simply do not have sufficient abstraction power, leading to class structures with overspecified detail. As an example, I was looking the other day at a paper on extracting models from Java bytecode. This process was essentially trivial *EXCEPT* for figuring out when some combination of fields and types was modelling a collection class, and this required some very large percentage of the code in the extractor. SK> I do think that the types I described are useful even though they SK> don't solve the maintenance problem. I agree entirely. SK> I think of an "otherwise" function as a combination of a "threader" (to SK> use your term) and a "collector". The types are useful precisely because SK> they suggest appropriate (and often natural) operations to use here. SK> Threaders follow principally from inherited types, while collectors are SK> induced primarily by synthesized types. (Eg: in the bus stop example, the SK> synthesized type would be either a "set" or "list" of Person, depending on SK> how we wanted to treat duplicates. The natural collectors would then be SK> union and append, respectively.) Sounds interesting. I'd certainly like to see how this works out. SK> If a traversal needs to switch types, I think the traversal should SK> simply halt at that point, and a new traversal should take over. The SK> type represents the coherent action being performed. The switch in SK> types suggests a switch in action. I believe this is a design SK> principle that will reduce surprises. You might have something here: adding a field with a new synthesized type should be a warning that something screwy is going on that needs attention. Separating the inherited type from the synthesized type is useful: different classes will often have different inherited types (eg programs vs expressions), but the synthesized types are often the same in a given example. Now, at the beginning of your message you wrote: SK> It seems we are speaking the same language, but we disagree on the SK> likelihood of surprises. As you say, this is an empirical matter. But SK> more on this below. As I said, my intuitions are based on my fairly narrow experience. I really appreciate your insights. But what is the "more on this below" you refer to? Do you have or know of any empirical data on this? --Mitch From shriram@cs.rice.edu Tue Mar 16 20:17:13 1999 From: Shriram Krishnamurthi To: Mitchell Wand Cc: lieber@ccs.neu.edu, johan@ccs.neu.edu, dougo@ccs.neu.edu, lorenz@ccs.neu.edu, matthias@cs.rice.edu, mira@informatik.uni-siegen.de Subject: Adaptive Programming and functional combination Hi Mitch, I think your comment about the lack of abstractive power of OO models is quite accurate. Two things I often miss are "natural" built-in constructors, and polymorphism. Your example is amusing, and it comes as no surprise that it should arise in the context of Java (which suffers from both shortcomings). > But what is the "more on this below" you refer to? > Do you have or know of any empirical data on this? I didn't mean to give the impression that I have hard figures. I based my comments on (a) involvement in software projects such as DrScheme and a theorem prover; (b) impressions gleaned from people who work closely with OO languages on a daily basis, such as those I've met at FSE, ECOOP and Dagstuhl; and (c) inferences from reading about patterns and OO methodology. 'shriram