Design, Develop, Create

Friday 30 September 2011

"Designing software is an exercise in managing complexity"

It's hard to disagree with observations and statements that have the ring of truth to them. These two essays say something true about the work of writing software. I take the work of writing software to include everything that contributes to the concrete production of an information system, production of the system as a concrete source code listing and byte coded thing through to configurations, usages, practices, aesthetics, etc.

So taking this broad characterisation of software to be Reeves's subject matter what is he stating and is it important? If we accept what he writes to be part of the truth of managing systems development, how does this understanding inform how I manage and how I work with technology?

REFERENCES
1. Reeves, J. W. (1992) What Is Software Design? C++ Journal. (link)
2. Reeves, J. W. (2005) What Is Software Design: 13 Years Later. C++ Journal. (link)

Thursday 29 September 2011

Problem posting to Blogger from different browsers...

A number of people have had problems posting comments to Blogger from either Google accounts or OpenID accounts.

I reproduced the problem and resolved it on Safari, Firefox and Chrome by clearing the browser cache. I think the problem arises from the way Google handles multiple logon identities in the browser.

The 2009-10 ASE class cantilever exercise



The ASE class were my guinea pigs for the first version of the cantilever exercise.

The MSD class 2010 working on the cantilever problem

The following slideshow records The MSD class 2010 working on the cantilever problem.

materials pack 1
The Guindon design exercise says something deeply important about the dynamics of design work in teams addressing partially specified problems.

Student solution to creativity exercise.

One of the students from 2010 captured her group's solution which was probably the most entertaining of the whole classes.

Group Creativity In-class Project from Sileg on Vimeo.

Tuesday 27 September 2011

Respect for rigour in software development

Respect for rigour in software development. The way to software you can trust your life with is through rigor and control. In Fishman's (1996) report we are provided with something of an overview of the situation, the resources, the working environment and the demands being satisfied by a software team responsible for developing the on-board flight control programs for the Space Shuttle.
"Consider the stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors." (Fishman,1996)
"the shuttle software group is one of just four outfits in the world to win the coveted Level 5 ranking of the federal governments Software Engineering Institute (SEI) a measure of the sophistication and reliability of the way they do their work. In fact, the SEI based it standards in part from watching the on-board shuttle group do its work. (Fishman,1996)"
But if rigor can be good, why do CMM, CMMi and other control frameworks have such a bad reputation in the profession?
"I am convinced that most organizations using the CMM are still entrenched in a default waterfall model mentality. I won't lay blame on the model itself, for I am aware of some process improvements made within a CMM context that were very much based on a modern, iterative approach to development. But this enlightened interpretation is not the norm." (Royce, 2002)
Is rigor only possible with frameworks and management control? Are other kinds of rigor and control possible? If there is a common problem what is it? At what point does more 'stuff' become the problem rather than the solution to the problem?

Commentary on 'the Right Stuff' on the C2 wiki (link).
And the backstory to waterfall, Winston and Walker Royce (link).


References
1. Fishman, C. (1996) They Write the Right Stuff.
2. Royce, W. (2002) CMM vs CMMI: from conventional to modern software management.

Thursday 22 September 2011

Theories of Organisation

THEORIES OF ORGANISATION
What theories underpin contemporary coordination and communication approaches? How do are these theories arrayed and structured organizationally, what tools, and knowledge are assumed and necessary to manage the implementation/delivery mechanism? How might we interpret or theorise the underlying processes taking place in this cycle of learning and adaption, because of course technology and systems don’t evolve of their own accord. If we can assume that the organizational system for high tech development is itself a knowledge intensive system, with some (but only partially) immutable structure, then we can employ organizational theory to make sense of (and perhaps manage) these systems.

MAKING SENSE OF ORGANISATION
Organisations are a kind of social order that we ourselves take part in constructing to give shape to special forms of collective action. As Max Weber observed, take away the people and organisations are nothing (Weber, 1949). However sociologists remain divided on the topic of the structural status of organisations and even society. We generally treat social structure as if it is real. Indeed there may be as yet unidentified physiological/psychological mechanisms bases for social patterning, they remain at the moment however, mere conjecture. But even if the human mind produces society and social structure through some as yet unknown intricate biological mechanism, the putative extrapolation of such a function from the individual up to the level of organisational forms, indeed to cultural, social and societal forms (Compte, ), is still the realm of science fiction rather than of science.

Whatever about the status and debates in organisation theory and sociology we still want (and need) to understand and make sense of organisations, especially as we are all in various ways involved in their production. Understanding how we go about doing this is important because our very understanding is in some way bound up with our experience of organisations. The ways we make sense of organisations attunes us to how we interact with others. The language we use, the ways of thinking about events and contingencies encountered from day to day predispose us to ways of interpreting work and our working lives.

METAPHOR
Assuming that there is as yet no unambiguous physiological or functional mechanism underlying the phenomena of social organisation; how then do we make sense of organisations even as they appear to surround us and constitute the fabric or our organisational lives? Metaphor is one way. The language and concepts we use in everyday speech govern to some extent our perceptions, distinctions and decisions. Metaphor is one way of interpreting and making the complexity of everyday situations comprehensible. A metaphor can represent a working hypothesis, a theory of what is happening, how it happens, who is involved etc. Metaphor is an approach we all use, almost without thinking about it, to interpret and simplify everyday realities. Lackoff & Johnson (Lakoff and Johnson, 1980) argue that organisational realities are constructed in language, and that metaphor is a pervasive and necessary element of this.
“Our concepts structure what we perceive, how we get around in the work, and how we relate to other people. Our conceptual system thus plays a central role in defining our everyday realities.” (Lakoff and Johnson, 1980)
“Metaphor pervades our normal conceptual system” (Lakoff and Johnson, 1980)
The prevalence of metaphor and its role in constituting organisational meaning may be a crucial instrument for interpreting and managing social interaction and (potentially) organizational design. Skilled and effective managers and professionals appear to have an innate ability to ‘read’ organisational situations, analyse and diagnose them by forming theories of what is happening and devising strategies to address situations (Morgan, 1986). There may be a direct association between a manager’s abilities to ‘read’ situations and their operating theories of organization. A metaphor lends a particular view or insight into a situation, one that ‘frames our understanding’ (Lakoff and Johnson, 1980) of what is going on; it is an operating theory of the situation.

Gareth Morgan (1986) offers eight images of organisations for interpreting and diagnosing conditions and situations (Table below).
Table: Eight Images of Organization (Morgan, 1986)


Lets look at three of these images (organisations as Machines, as Organisims, and as Brains) as ways of interpreting organizational systems and thinking about how to organise systems development.

ORGANISATIONS AS MACHINES
“When managers think of organizations as machines they tend to manage and design them as machines made up of interlocking parts that each play a clearly defined role in the functioning of the whole.” (Morgan, 1986)
The machine metaphor has become so pervasive that it is often simply unquestioned. Mechanistic models are one of the foundational concepts of organization theory although the software development industry appears at times to have a special attachment to it.

Functional mechanistic organisation design may be a direct analogue of the organization’s dominant product architecture or be based on process specialization. Product analogue organization maps the macroscopic structure of the development firm along functional lines to the functional organization of the product. Process specialization is where rather than the organisation's product determining the design of the organisation, the organisations processes (chains of activities) determine the structure of the organisation. These processes (e.g. maintenance, new product development, consulting) govern the divisional structure of the firm even though at the micro level (in sub groups or teams) individuals may still apply a mapping between the product design and their own roles and responsibilities. For example specialist engineers will own specific code modules for new development, fixes etc.

ORGANISATIONS AS ORGANISMS
“The problems of mechanistic visions of organisation have lead many organization theorists away from mechanical science and towards biology as a source or ideas for thinking about organization.” (Morgan, 1986)
The metaphor of the organisation as an organism presents a view of the organisation as having needs and existing within the wider environment that impacts it. The organism itself consists of interdependent organs and is situated within an environment or ecology. It may have relations with other organizations and belonging to a type or species.

Certainly the ideas of biology have also, like those of mechanisation, filtered into popular discourse. Biological thinking has become one of these background assumptions constituting modern thinking. Organisation theorists have adapted the idea of an organism to that of organization; existing in an open system, adapting to its environment, interacting with other organisms, and applying the notion of fitness, surviving, and evolving over generations. The organism metaphor conveys ideas of a life-cycle, health, reproduction, predators, prey, food, waste, etc.

ORGANISATIONS AS BRAINS: LEARNING
Organisations as mechanistic or as organisms assume and encode very specific models of learning and knowledge. Mechanistic feedback control loops are goal driven error minimising functions. The classic negative feedback loop is the underlying theory for many organisational principles. The Plan Do Check Act model (ISO, 2000) achieves product conformity by first planning, acting, detecting the error between what was desired and what was achieved, and then adjusting (Figure below). The organisational system applies an adaptive approach to achieving convergence with some defined objective or goal. Such cybernetic or feedback systems in organisations apply this kind of action/learning method to dealing with uncertain situations.

Figure: PDCA single loop learning of the Demming cycle

Single loop learning is characterised by negative feedback, assessing the difference between what was desired and progress to attaining it. The error or difference is measured and then minimised, further action is dictated and the process continues until the goal is achieved. This kind of cybernetic feedback loop may be explained by the example of picking up a pen. In a sense we pick up an pen by ‘avoiding not picking it up’ (Morgan, 1986: 85). The argument goes that iswe make a guess or approximation of where the pen is then gradually adjust our hand’s position towards it.

PDCA is a kind of ‘single loop learning.’ (Argyris, 1977) A single loop learning organisation designed around PDCA is good at producing and refining well defined products efficiently, predictably, and within high quality tolerances (think Six Sigma) but they may not be as effective at responding to changing requirements and dynamic competitive environments. One problem with simple learning models is that they can be inherently incremental, focused on perfective rather than more radical adaptive learning. The PDCA or the single loop learning model is perfective rather than adaptive.
Throughout the 1990’s Kodak “kept making the process of manufacturing and distributing chemical-based film more efficient instead of devoting attention to making the shift to digital photography” (Amabile and Khaire, 2008)

Put another way, the single loop learning model doesn’t tell us much about adapting the organisation to changing circumstances, new goals. An intelligent person for example, situated in a particular context treats learning in a fundamentally different way. We can conclude that as with individuals, organizations can step out of single loop cybernetic goal directed action by reflecting. According to Morgan, overcoming the organisational limitations of single loop learning involves: Institutionalising review, challenging norms, challenging policies, challenging operating procedures, reporting on change in the operating environment, and encouraging on-going debate, encouraging innovation (adapted from (Morgan, 1986)). All of which will however generate organisational tensions if single loop learning is institutionalized through the organizations structures and procedures.

Wednesday 21 September 2011

Systems Development Life Cycle Notes

Life Cycle Archetypes
A simple definition of the classical Systems Development Life Cycle (SDLC) depicts high tech systems development in terms of 4 main activities or phases (Raccoon, 1995). Raccoon’s insight was to interpret other life cycle models in terms of the SDLC. Raccoon’s four life cycle archetypes are easily extended as variations on the theme of the SDLC as activities rather than as a sequence. If interpreted as a strict linear sequence the SDLC equates to the waterfall, if as overlapping phases it approximates the ‘V-model’, the spiral life cycle as lingering, and chaotic as all-at-once or code-and-fix. As Raccoon suggests, life cycles are just our own perspectives “on the state of a project rather than any essential truth about the state of the project” (Raccoon, 1995)
1) Requirements Analysis
2) Design
3) Implementation
4) Maintenance.
The waterfall model can be presented graphically in terms of SDLC activities as shown (below). The effort associated with these activities (or phases) can be plotted over time to create a shorthand representation of any life cycle in terms of the effort spent on these activities over time.

lifecycle_waterfall

lifecycle_sashimi

lifecycle_regression

lifecycle_chaos

Watefall

Sashimi

Regression

Chaotic

A life cycle can in principle be presented in similar fashion, for example the Rational Unified Process depicted below conforms roughly to Raccoon's ‘Regression’ archetype.
ruphumpback

References:
Raccoon, L. B. S. (1995) The chaos model and the chaos cycle. ACM SIGSOFT Software Engineering Notes, 20, 12.

Exercise: Systems Development Life Cycle Analysis

Objective:
To translate descriptions of different systems development life cycles into realistic/probable milestones and actionable activity timelines; as templates for the purpose of choreographing or managing teams.

Preparation:
Allocate at approximately 30” to run this exercise. 5” setup and briefing. 15” complete the exercise instructions below. 10” debriefing. This exercise can be carried out individually or in small groups. No special room/space requirements. A document projector is desirable.

Material:
Provide copies of this instruction sheet and additional blank sheets of paper.

Instruction:
Generate a milestone calendar or activity/time diagram using (at least) the activities of the generic SDLC, for one of the following life cycle models.

  1. Waterfall.
  2. Spiral development.
  3. Or any other life cycle/method you are familiar with.

Outputs:
A single A4 sheet of paper upon which each student/group creates their own annotated diagrams for the life cycle model, e.g. a timeline of phases/activity for a project or iteration, a map of business function/activities over time, a consolidated diagram. Write your name(s) on the sheet.

Learning Outcomes and Reflection:
At the end of completing the exercise the tutor can gather the diagrams, group and display them during discussion using a document projector. Alternatively select student/groups to show, talk about and explain their diagrams to the class.

Practical Aim:
Identify the key characteristics of the listed software production and systems development lifecycles.

Knowledge Aim:

  • Describe and discuss current and emerging management approaches to systems development in general.
  • The overall goal of this exercise is to explore how you would operate or organise your team's activities over time based on one or other lifecycle.

Cross reference: Systems development life cycle notes

Life cycle considered harmful - stop - I want to get off

Let's not fool ourselves into thinking that our grandparents programmed in a world unfettered by software development lifecycles and the like (Gladden, 1982; McCracken & Jackson, 1982). What appears striking from these accounts is that the authors appear to have established their careers as programmers without the 'benefits' of standard life cycles or packaged methodologies.

It may be of interest to reflect on the careers of these early critics of software development life cycles. Michael A. Jackson (link) went on to develop JSP (Jackson Structured Programming) and subsequently JSD (Jackson System Development). Both design methods for system development rather than lifecycles or management frameworks. McCraken's career also flourished (link) in industry and academia. I have no further knowledge about G. R.Gladden aside from his/her role as quality assurance supervisor at Honeywell's building services division in 1982.

REFERENCES
Gladden, G. R. (1982) Stop the life-cycle, I want to get off. ACM SIGSOFT Software Engineering Notes, 7, 35-39.
Mccracken, D. D. & Jackson, M. A. (1982) Life cycle concept considered harmful. ACM SIGSOFT Software Engineering Notes, 7, 29-32.

Monday 19 September 2011

Storyboarding ideas

Why should I care about what you've got to say?
Storyboarding is a technique I can use to help craft my message (Reynolds, 2008). Going about the business of presenting ideas has to be seen as a process. It's a creative process that rarely proceeds in linear, sequential fashion from initial concept through to completed work. The problem with software tools like PowerPoint is just that, they are tools, part of my equipment but not the source of my inspiration nor necessarily the subject matter. That said the tools are great aides for producing 'the work' but I need to include all the equipment I'm going to use because it's all part of the process and therefore necessary and relevant: sticky notes, whiteboard, back-of-a-napkin, sheets of paper, and software tools.

Foremost I should know what my message is. In this case, I want to convince others that storyboarding is a great way of structuring a persuasive narrative. I also want to link this to the idea that the media I use is merely an adjunct to the the narrative, even when I capture and distill my story in a close-ended format like film. What I mean is by this is that the narrative still needs me (and you the audience) to interpret it.

Garr Reynolds describes 'crafting the story' as a process, a process that takes place over a number of steps (Reynolds, 2008). I'll use the phrase 'categories' rather than steps. The process of crafting the story starts from a 'core message after which we branch into a mixed sequence of activities that I'll paraphrase from Reynolds as follows:
  • Brainstorming
  • Grouping & identifying the core
  • Layout and organising
  • Dry run and re-organising
Implicit in the process is its iterative, non-linear nature.

Let's look at "The Learner's Journey in Practice" by Brian Sawyer to illustrate the narrative of a story and an approach to structuring it. Sawyer presents a case of storyboarding with a colleague (Michael Milton) to outline the detail of a book chapter. They structure the chapter around a learner's learning process. They start from a basic linear narrative ploy, learning as a journey with a beginning, middle, and end. The learner they envisage needs to cover a number of major points and the major points are interspersed with supporting subtopics. They then create a scenario, a "learner's journey", to overlay the storyline. Sawyer then uses the idea of an actual reader undergoing his or her own learning experience; feeling the peaks and troughs of accomplishment, the 'oh crap' valleys and the 'I rule' moments. The scenario becomes a narrative tool to refine the chapter content, order, and presentation.

Sawyer's linear story is just one way of depicting what Reynolds calls layout. But how does the scenario work? The story is the simple linear sequence but the narrative is what they construct around the bare facts of the story. The narrative sketches how someone (a generic audience) encounters the facts as they are presented or made available 'in order'. Perhaps most important and implied but not explicit is that the rough notes, the storyline and the narrative structure are also necessary tools and technique for communicating ideas these. In the first instance Sawyer might be working alone but still putting ideas down and re-engaging with them, reorganising them. This process of capturing, organising, reorganising is a simple compelling account of what goes into presenting ideas but a lot of work has taken place prior to this stage; the goal of the book, deciding what the chapters should cover, how the chapters relate to each other etc.

RESOURCES
The following tutorials from UCD's School of Information and Library Studies (now titled the UCD iSchool) may be of interest (note that JayCut, the Blackboard Wiki, and other systems they describe are no longer available).


Footnote:
Storyboard techniques for software projects

REFERENCES
Reynolds, G. (2008) Presentation Zen: Simple Ideas on Presentation Design and Delivery, Berkeley, CA, New Riders.
Sawyer, B. (2009) The Learner's Journey in Practice. (blogs.oreilly.com)

Creativity: Wile E. Coyote and Roadrunner

Robert Fabricant positions creative interaction as the crucial dynamic within groups of people solving problems. That is, creative interaction as a messy, chaotic, conflictual - but occasionally productive - dynamic between people who may or may not get along, with customers who may or may not get exactly what they want or even need, and making things you didn't know you were going to make before you made them (really).

designmind.frogdesign.com

Wednesday 14 September 2011

Big Ball of Mud

It might be said that any software codebase and architecture initially resembles spaghetti code by being incomprehensible at first to any but an acolyte deeply familiar with its design.

The Big Ball of Mud architecture pattern was proposed by Foote and Yoder (Foote et al. 2000) as an 'anti-pattern' in software design and an inevitable consequence of the forces experienced by development. Development takes place within an environment of contrary forces and time pressure that imposes restrictions, constraints etc. on 'proper' design. For example, foundational design concepts such as coupling and cohesion of code modules are gradually broken and so architectural integrity degrades over time due to the ad hoc nature of evolving needs, requirements, and bug fixing. Boehm claimed that both ends of the lifecycle spectrum, ‘Code and Fix’ and Stagewise models end up producing unmanagable code or programs that fail to address the customer's goals (Boehm 1988). Foote and Yoder take an alternate view, that life cyle isn't the problem, the problem is intrinsic to code that succeeds and therefore needs to be maintained over a long time. They suggest that the BBM pattern is present as entropy, an attractor to disorder that imposes a constant tension on code but one that can be resisted through the practice of Refactoring.
"A certain amount of controlled chaos is natural during construction, and can be tolerated, as long as you clean up after yourself eventually. Even beyond this though, a complex system may be an accurate reflection of our immature understanding of a complex problem." (Foote & Yodder, 2000)


On the mysterious mound builders of Mountain View.

Brian Foote observes that in the years since he and Yoder wrote the article, "that no one has ever challenged our premise. No one has ever said 'no it isn't the most frequently deployed architecture out there', 'it isn't what we're all doing'" (Foote, 2007: 17'29")

One of the dynamics that produces BBM is turnover. "All too many people get onto projects, beat the heck out of the code, and move onto the next project. The code 'dies', the code is unmaintainable, nothing else can grow there, the enterprise founders, the end of the story." (Foote, 2007: 30'35")

The messiness of the relationship between code, architecture, and design may again be an intrinsic facet of software. Richard P. Gabriel (link) is credited with the counter-intuitive adage that 'worse is better' to characterise the dynamics of software acceptance. Subtly it can be understood from two perspectives, the developer and the user. From the developer's perspective worse code may produce better results for the user. From the user's perspective less features (worse) may be better because simplicity is what the need rather than what they want.

REFERENCES
Big Ball of Mud: Foote, B. & Yoder, J. (2000) Big Ball of Mud. IN HARRISON, N., FOOTE, B. & ROHNERT, H. (Eds.) Pattern languages of program design 4. Addison Wesley. (version online)
A wiki discussion on one of BBM's topics at Ward Cunningham's c2.com (link) - photo credit JohnLennon shovels on the spaghetti in cover art from the Magical Mystery Tour.
Foote, B. (2007) Big ball of Mud. Google Tech Talks.

Tuesday 13 September 2011

Motivation

Consider work that involves genuinely creative and inventive aspects but work that also by necessity is the product of group interaction and multidisciplinary involvement.

Dan Pink's compelling (with RSAnimate's engaging video/graphic style) piece on motivating humans, intrinsic vs extrinsic, a (now) classic web presentation that must be seen (video on youtube but produced by www.thersa.org).

Question:
So how does compensation structure perform as a determinant of team performance? Put another way, how do teams perform as collaborative units under differing reward structures?

Monday 12 September 2011

Creativity and the Design Process

Creativity and production of digital goods: managing creative intellectual teamwork
The results or outcomes of creative processes are unique and sensitive to context. This section reviews practical strategies for unleashing the creative dynamics of teams in digital production. Creative performance on teams is shown to be affected by group relations, structure, composition and skill. Therefore management has a role in cultivating the conditions for the creative dynamic to occur by supporting divergent and convergent thinking, producing a (usually) productive process of problem/idea/solution generation.

PERSPECTIVES ON KNOWLEDGE, LEARNING, AND CREATIVITY
"Seymour Cray described how his little company, located in Chippewa Falls, Wisconsin, had come to build what are generally acknowledged to be the fastest computers in the world... Cray said that he liked hiring inexperienced engineers right out of school, because they do not know what’s supposed to be impossible."
(Kidder, 1981)
If I cannot control an individual’s creative potential how is it possible to manage that of a team? It seems possible that two different processes are in play, collective and individual, and that both processes are interrelated in certain ways. E. Paul Torrance defined creativity for his groundbreaking studies of creativity in children as:
"The process of becoming sensitive to problems, deficiencies, gaps in knowledge, missing elements, disharmonies, and so on; identifying the difficulty; searching for solutions, making guesses, or formulating hypotheses about the deficiencies; testing and retesting these hypotheses and possibly modifying and retesting them; and finally communicating the results."
(Torrance, 1965)
Torrance described two main activities in the creative process: divergence and convergence. Divergence described ideation, extrapolating and growing ideas; convergence described judging and selecting ideas, reducing the alternatives being considered. People displayed different proficiencies for purely original thinking versus elaborating on existing ideas. The creative process consisted alternatively of divergent and convergent thinking. There are also two broad approaches to understanding creativity; on the one hand, creativity as an individual's own cognitive process (Treffinger, 1995), and on the other hand, organizational/contextual analyses of creativity or innovation (Hargadon and Bechky, 2006). The brainstorming captures elements of both individual and team creativity whilst it structures the two activities. The suspension of judgment allows for elaboration and extrapolation of ideas, while a returning focus on the problem allows for periods when ideas can be tested and selected.

Certainly creativity is largely an individual cognitive activity. Indeed the ‘ex nihilo’ generation of ideas may be a necessary underlying process of individuals during a collective design or problem solving activity. However cognitive psychologists conclude that individual creative processes are more often processes of analogical reasoning applied to make sense of new situations rather than gestalt insights. Therefore the individualist perspective overplays the role of the inventor in the process of creative production while innovation studies in particular, often emphasize the importance of contextual factors in successful innovations but underplays the role of ‘supraindividual creativity’ (Hargadon and Bechky, 2006).

However there are interesting dynamics surrounding design which unfold over the life of a team that suggest a strongly collaborative collective aspect to design and learning in general. Creativity is can be defined, in terms of team outcomes, simply as the “recombination of existing ideas” (Hargadon and Bechky, 2006) to produce a novel solution or address a problem. From this perspective the phenomenon of interest will be apparent in groups through interaction and behaviour.

SIX CREATIVE WORKPLACES
"What turns collections of creative individuals into creative collectives, where particular interactions yield creative insights, yet those insights cannot be attributed to particular individuals?"
(Hargadon and Bechky, 2006)
Attempting to answer this question drove longitudinal study of professional service consulting firms in the business of developing ‘novel and valuable solutions’. The study was designed to understand how collective creativity takes place in groups and therefore addresses a gap in our understanding of the links between creative problem solving and innovative production in organizations. Hargadon and Bechky's study relates to digital and software development because it can inform how we might better understand creative processes in software development teams. 

The findings are based on six in-depth case studies of highly creative industry workplaces. The research looks at group members’ own reflection and sense making of their experiences working in groups involved in creative production. All six firms in three industry areas worked directly on problems that required creative solutions. The view of collective creativity presented is plausible and actionable by others. If the practices can be introduced they could then be regularized in policy or supported by management. 

Where Do Collective Creative Processes Take Place? Hagridden and Bechky’s unit of analysis is group interaction and design behaviour.
"this perspective recognizes the fleeting coincidence of behaviours that trigger moments when creative insights emerge. ...insights that emerge in the interactions between individuals."
(Hargadon and Bechky, 2006)
Their analysis identifies a model of collective creativity consisting of four dominant interrelated activities: help seeking, help giving, reflective reframing, and reinforcing. They draw on Weick and Roberts idea of mindful engagement in social interaction that shapes group or collective cognition.
"Mindfulness describes the amount of attention and effort that individuals allocate to a particular task or interaction. Participation in group interactions, as a result, becomes a product not of membership or presence within a group, but of the attention and energy that an individual commits to a particular interaction with others in the group."
(Hargadon and Bechky, 2006)
HELP SEEKING
Informal events and formal processes were identified as important enabling factors to initiate problem solving. Chance meetings in halls were cited as were organizational norms for brainstorming and problem solving.
  • Design Continuum used formal brainstorming sessions.
  • HP’s SPaM group and IDEO held weekly meetings to openly discuss current projects and problems.
  • Boeing’s Ops Tech group met monthly but also informally more frequently.
"The simple practice of holding regular review and brainstorming meetings was an important enabling factor was were ‘ad hoc meetings,’ ‘hallway conversations,’ and ‘tapping into personal networks."
(Hargadon and Bechky, 2006)
The absence of social costs or sanctions for asking help after failure, and organizational cultures that did not stigmatize people for seeking help for problems or failure are powerful enablers to reinforcing help seeking behaviour.
At IDEO "blame for any particular design failure depended on whether the engineer had asked others for help or not: If they had not sought help, then they would be held individually responsible." (Hargadon and Bechky, 2006)

HELP GIVING
The counterpart to ‘help seeking’ behaviour was ‘help giving,’ without which there could be no collective creative interaction. ‘Help Giving’ also had to take place and in a timely manner. There were obstacles to help giving. The more experienced people were often very busy. There were often institutional constraints around accounting for people’s ‘billable time’ and you can’t always get the obvious person involved. Help seekers would resort to people who were available and accessible instead of going to the top.
"You talk with whom you can. You explore until you find people in the firm that are accessible, near enough in the time frame to talk about it." (Hargadon and Bechky, 2006)
"There were certainly times when, for example, solicitations for help arrived as clear questions and could be easily returned with equally clear answers." (Hargadon and Bechky, 2006)
However while the process of formulating and expressing the problem in documents can often help resolve the issue, email and document mediated interactions could be useful but lost immediacy and clarity.

REFLECTIVE REFRAMING
Reflective reframing is a process of recasting problems or solutions from one field into another. The actual process or act of expressing the problem with another and engaging with them in a ‘brainstorming session,’ ‘ad hoc meeting,’ or ‘hallway conversation,’ was the key interactive dynamic that represented the formation of the creative collective.
"Within such interactions, introducing an alternative frame – and reflecting upon it – makes new aspects of the situation salient to other participants, prompting them to view the relevance of their past experiences in a new light." (Hargadon and Bechky, 2006)
This was the experience of expressing the problem with others engaged mindfully in the interaction. It was a joint process of seeing solutions in terms of past experience or reframing the problem from a different perspective.

REINFORCING
Reinforcing is the normative personal and social process of feedback and recognition (both good and bad). Reinforcing plays an important role in making individuals aware of the activities which feed into the creative process and it predisposes individuals (positively or negatively) to engaging with the process again. The set of behaviours will also be reinforced if individuals have positive experiences when they engage in the process of help seeking and help giving. Reinforcing influences shape how individuals may become more or less open to future creative interactions, capturing in some sense how people remember these experiences and value them. Reinforcing behaviour becomes a matter of shared values and beliefs, the local organizational culture and that at large. Importantly they found that
"asocial means of enabling knowledge sharing do not encourage people to participate in joint problem-solving efforts." (Hargadon and Bechky, 2006)
CONCLUSIONS
It is evident, from research and experience, that collaborative creative interactions are complexly sensitive to issues of group interaction, relations, behaviour, practices, skill, group composition, experience (prior and together), and access to each other, time, materials and other resources. This list of factors is also unlikely to be exhausted. Performance of creative interaction is subject to the vagaries of situations, context, history and local culture. However there are some things we can say about creativity in teams; creativity is an emerging phenomena. The ideas and decisions made in collective processes emerge in unplanned ways. The quality of the ideas is a function of various group aspects alongside structural enablers like norms, culture, attitudes and process. Unquantifiable aspects may be as important as those we can manage; for example, ambiguity may be a necessary condition to allow misunderstandings to occur. Ignorance of prior limitations may allow the problem to be approached in a fresh light, unencumbered by pre-existing paradigms.

Atlassian thinking behind agile practice

(via Dagny) Atlassian present a video piece on their own agile development approaches. In these interviews with the employees they highlight the difference between agile & waterfall in a brilliant way. Many more videos supporting (atlassian.com)

Who is Kent Beck?

Kent Beck is a software programmer running a small consultancy business in Oregon. As he says himself, he divides his time between writing, programming, coaching and farming Goats. Kent was a SmallTalk programmer who attempted to re-think the essence of software development having been involved in attempting to recover some significant software project failures. JUnit is a unit testing framework that has since spawned the development of other testing frameworks NUnit etc. He has written several important journal articles and books, mainly addressing Extreme Programming, SmallTalk programming and JUnit, his most successful project.

Kent is a respected voice in software development's professional communities; taking an active role in discussions, speaking at conferences and sharing with other programmers these ideas for a better way to organise the work of programming for which he coined the label 'Extreme Programming.'

For more see:
Kent on Twitter
Kent's Three Rivers Institute
And the active Extreme Programming group on Yahoo (extremeprogramming@yahoogroups.com)

Friday 9 September 2011

Who is Barry Boehm?

Barry Boehm's career in computing and software coincided with the early evolution, definition and expansion of the discipline of computer programming in the 60's. His background in mathematics (primary, masters and Ph.D.) predisposed him, through his early programming experiences, to the need to attain algorithmic correctness, however his subsequent exposure to the production of relatively large programs, diverse programming languages, diverging computer architectures, human interface and usability considerations drew him to conceptualize of the general experience of programming as a highly subjective exercise in group communication.

On Wikipedia
Barry Boehm quotes

A modern interpretation of Royce

There are some points we can take from Royce if we interpret him generously:
Program Design Comes First: Analysis/Design (and code) is indeed the heart of programming, but is it realistic to expect these activities to reside mainly at the beginning of a project? Document The Design, and/or perhaps the architecture (whatever that might be) is desirable but in answer to the question of 'how much documentation?' Royce states
"[m]y own view is ‘quite a lot;’ certainly more than most programmers, analysts, or program designers are willing to do if left to their own devices. The first rule of managing software development is ruthless enforcement of documentation requirements." (Royce, 1970)
On this point Royce suggests that for a "5 million dollar hardware device, I would expect that a 30 page specification would provide adequate detail to control the procurement. In order to procure 5 million dollars of software I would estimate a 1500 page specification is about right in order to achieve comparable control." (Royce, 1970) p332  More ambiguously perhaps he mandates that (at least at the start of a project) "the documentation is the specification and is the design." It is clear that Royce assumes that the document communicates perfectly and unambiguously, that a document overcomes the problems of interpersonal communication (or its lack) by being clear and unambiguous. If we read ‘understanding’ instead of documentation and consider it to be present in ‘communication’ in the processes and practices of communication, of interpersonal interaction and sense making along with the source code and ‘documentation’ then perhaps Royce isn’t far from the mark.

Do It Twice, an insightful recommendation, that the software is actually a product of learning; learning what the requirements really are rather than what they were initially thought to be, learning how a design performs, how it interacts with the hardware and other software, learning how the software is actually used, timing, scale, performance.

Plan, Control, And Monitor Testing. Yes but, perhaps happening later as it does in his model, the model may collapse as bugs, problems and inadequacies leak back again to affect the requirements, design and analysis.

Finally Involve The Customer, perhaps the most insightful recommendation and perhaps one that we still find difficult to achieve.

Royce can be read perhaps as a radical, making recommendations not usually associated with this era, principles and practices which might be understood in Agile Development as emphasizing communication, design/coding as the central activity, responding to change, refactoring design, visibility, and on-site customers. At the end he arrives at a process model for software development, an approach which is essentially iterative, involves customers, and is dominated by strategies to facilitate communication with all involved.

While Royce’s paper was influential in the move towards methodologies and understanding and developing principles for the organization of software development teams, it also (sadly) said very little about the practicalities of organizing teams and actual people. For this one must turn to Brooks (1995 (1987)) and others who followed.


REFERENCES

Some questions on systems development

Question: Do the objects and practices of the software engineering domain propagate elsewhere? Are they reproduced in other professions and settings?

Questions on Royce's article:
  • Who are the ‘designers’ if not the ‘programmers’?
  • Comment on Royce’s assertion to ‘document the design’
  • How does Royce remove task leakage in his modified model?
  • Does his modified model really succeed in overcoming leakage?
  • What does the direction of Royce’s diagram arrows represent?
Questions on Boehm's Spiral Model:
  • Does Spiral end up treating development as a game, with the goal being to pass the ‘review,’ lead to short term perspectives with implications for investing away from long term goods like training and discovery?
  • Can the spiral model be used to develop/maintain a mature product/project?
  • Is a single generic life cycle useful?
  • How might successive generations of a product undergoing continuous development and delivery in new releases be managed using spiral?
  • Does spiral address the activities of production/programming (analysis, code, fix)?
  • What are underlying drivers of increasing complexity of high-tech products?
  • Does spiral that programming is an intrinsic process of analysis, design, coding, fixing, testing ad infinitum?
Questions on Raccoon's Chaos Model:
  • Is Raccoon’s analysis insightful or useful?
  • How might we act differently on the basis of this Raccoon’s presentation of the work of design and coding?
  • Taking Raccoon’s idea of recursive development seriously, can we also assume the same approach apply to testers, QA, consultants, and product owners, even customers themselves?
Questions on Process Frameworks:
  • Are software process frameworks The challenge for systematic (and systemic)?
  • Do process frameworks reinforce implicit linearities in conventional production?
  • Can process frameworks support iterative processual modes of working?
Questions on ISO9001
  • Can it can be mapped onto all the different life cycles in use in the software industry, e.g. waterfall, incremental, evolutionary, spiral life cycle, etc?
  • Should you deliver something that hasn’t been tested, or fails the tests?
Questions on Agile development
  • Which half of the list of XP Rules is already accepted as general programming best-practice?
  • Isn't Pair Programming an unnecessary doubling up of resources?
  • Why should a development team adopt Collective Ownership? Surely it is a recipe for out-of-control code?
  • Comment on the following: 
    • Continuous Integration of developer code will produce a bug factory, i.e. code perpetually broken and in an indeterminate state. 
    • Refactoring is a way of saying we haven't got the architecture right (do we even have an architecture?)!
    • Agreeing to 'sustainable pace' or a 40 hour week will slow product development down.
    • The XP Planning Game removes control from designers.
  • Royce’s proposed remedies to the waterfall model (Royce, 1970) can be read as a radical approach to refocus on principles and practices rather than structure and method. Is it reasonable to interpret Royce in terms of the Agile movement? Royce’s ‘5 Steps’ suggested:
    1. Program design comes first (code is design)
    2. Document the design
    3. Do it twice
    4. Plan, control and monitor testing
    5. Involve the customer.
Some Questions on Creativity
  • Consider Sutton’s recommendations, are they irresponsible?
  • Is a creative culture following Sutton doomed to self-destruction?
  • Does it make sense to conceive of a stable self-sustaining version of Sutton’s creative culture?
  • Can a creative development culture coexist with a general production culture?
  • From Katz; how does compensation structure perform as a determinant of team performance? 
  • How do teams perform as collaborative units under differing reward structures? 

Managing teams

COLLABORATION IN TEAMS
The social dynamic of software design work in teams is beginning to be understood as an intrinsically creative process. Creative teamwork is often difficult to pin down however as it may manifest unexpectedly and in unplanned ways. Interpreting high tech design and development as a creative process is the first step to suggesting approaches to structure and manage conditions to better enable productive team centred design and production (Ancona and Caldwell, 1990).
Contemporary innovations in software process knowledge reflect a renewed emphasis on the importance of coding (Sharp and Robinson, 2004, Mackenzie and Monk, 2004), leading us to conclude that co-design activities of software development present an empirical moment of central concern to organizational studies of software product development. Software design in teams is an inherently social activity (Winograd and Flores, 1986, Suchman and Trigg, 1996, Dittrich et al., 2002, Weinberg, 1971) but one which seems to constantly slip from management and control.

Managing Creative Interaction
The creative problem solving process involves generating waves of diverging and converging ideas. The process of productive interactive problem solves some key practices and joint understandings.
  • Focus
  • Suspend judgment
  • Build on ideas
  • Personal safety
  • Serial discussion
The capability to solve problems creatively and to engage in creative forms of collective production is dependent on a number of distinctive behaviours that become embedded in the institutional setting (Hargadon and Bechky, 2006) p485.

  • help seeking
  • help giving
  • reflective reframing
  • reinforcing.

While the unfolding process itself involves distinctive transitions.
Brainstorming

Proficient teams of creatives will discuss, negotiate and establish group guidelines for brainstorming and other solution generative activities.

Design philosophies and development methodologies are rarely applied in a formulaic manner, rather they are used to initiate and facilitate the organization of design thinking, to better deliver usable computer systems supporting work and other activities.


TEAMS: MOTIVATION, REWARD, AND COMPENSATION
Collaboration on teams is evident in how team members teach each other and learn from together to overcome problems faced by the team as a whole. However peoples’ behaviour may be conditioned in part by the reward structures of the organization. Can reward or compensation structures have an impact on people’s propensity to teach and their motivation to learn? If so what reward structures support this behaviour? Is compensate or reward anti-ethical to teamwork and collaboration?

Many reward structures have cultural aspects (motivated by personal and professional beliefs and values) but compensation also plays a part. The right compensation scheme can at the very least impose or remove some of the obstacles to cooperation, collaboration and teaching/learning/creative processes taking place between team members. In an experimental study Nancy Katz experimented with 100 participants in a Bolo exercise as used by US Army Research psychologists (Katz, 2001). 35 three to four person teams played a war game simulation against the computer, capturing refueling bases (illustration below).
From Scrapbook Photos
Figure: Credit: Atari Battlezone. Bolo is a networked multi-user, real-time tank battle simulation.

The simulation created a number of common conditions; dispersed information, time pressures, easily evaluated performance, undifferentiated member roles, and the need for collaboration. The following compensation structures were employed and group performance assessed (Table below).
Table: Compensation schemes trialled in Bolo exercise (Katz, 2001)
From Scrapbook Photos
So how does compensation structure perform as a determinant of team performance? Put another way, how do teams perform as collaborative units under differing reward structures? Katz found that the hybrid ‘threshold’ schemes were best at encouraging behaviours that maximized the team’s performance. Both threshold schemes were associated with cooperative behaviour, increasing the likelihood “that people with greater skills will help their teammates become more proficient.” (Katz, 2001) p22 The group threshold motivated teaching behaviour on the part of the most talented team members,
“encouraging high performers to teach and share information.” (Katz, 2001) p22
The individual threshold scheme was a driver for individual learning effort, motivating low performers to learn and improve themselves. The ratio scheme, while better than both equity and equality, encouraged anti-group behaviour.
“most likely because it placed a cap on performance rewards and made team mates constantly concerned about their performance relative to one another.” (Katz, 2001) p22
Another aspect that might seem obvious but needs to be highlighted. There team performance is learnt over time. Team behaviour is sensitive to its experiences, how long it remains together, the relative learning trajectories of its members, the shifting levels of knowledge, skill and expertise which change over time.
“enough time for highly skilled members to teach their less skilled counterparts.” (Katz, 2001) p22
Without the time and opportunity to learn the processes underlying collaborative, learning the work just cannot take place.

Systems Development Themes

Introduction
An introduction to systems development

Perspectives on development
  • Economic Aspects
  • Life Cycle Concept
  • Frameworks for Control
  • The Rise of Agility
  • Parameters of Development
  • Theories of Organisation
  • Managing Knowledge
  • Creativity and the Design Process
  • Teams in Development
Concepts
  • Build the thing right (SDLC)
    • Requirements
    • Evaluation
    • Implementation
    • Maintenance
  • Build the right thing (Interaction Design)
    • Look
    • Learn
    • Try
    • Ask
    Exemplar cases
    • The Avalanche case
    • The Kongregate games case
    • IONA 1 case
    • IONA 2
    • The Scrum proposal case
    • Pixar 
    • Tracey Kidder's Soul of the New Machine (Tom West and Data General)
    And loads of practical exercises

    Wednesday 7 September 2011

    Eight work-style heuristics for creative system developers

    Jeremy Rose’s “Software Innovation: eight work-style heuristics for creative system developers” takes a rather novel spin on the key issues for systems development. A PDF version is available via bit.ly/bIRpr4 and a web version is online at softwareinnovation.pbworks.com/

    Thursday 1 September 2011

    The development 'life cycle' metaphor

    "Life is the state of ceaseless change and functional activity peculiar to organized matter."
    (Fowler and Fowler, 1929)

    What is a life cycle for a high tech development and why is the concept so pervasive?

    Implicit within the definition of a life is the idea of a beginning and an ending, birth and death, the span of a life, or in the case of technology, the duration of a project or product. The idea of a ‘life cycle’ is a relatively modern concept. Its published use commences in the mid 1800s when the idea of ‘cycles’ with connotations of lunar, diurnal, mechanical, of recurring events and events recurring in a chronological cycle. Some of the earliest references to ‘life cycle’ use it to refer to the entirety of a biological life described scientifically in which case life cycle depicts an entire biological life (birth, maturation, death) as an abstract sequence of events and transformations. From the early 1900s it is used next in sociology to characterise a person's life history. Its use then shifts to businesses from the 1950s and then on to manufacturing, product development, and marketing when it begins to encompass the relationship between the customer, products, the firm and activities within the firm. (OED, 2010).

    References:
    Fowler, H. W. & Fowler, F. G. (1929) The concise Oxford dictionary of current English, Oxford, At the Clarendon press.
    Oed (2010) life cycle, n. DRAFT REVISION Mar. 2009 ed., Oxford University Press. http://dictionary.oed.com/cgi/entry/50132966