Encyclopedia > Software engineering
Software engineering
Software engineering is the spectrum of applications that create economic and social value. By the year 2000, practitioners have built many successful and important systems, such as email, embedded software, graphical user interfaces, office suites, operating systems, optimizing compilers, relational databases, robotics controllers, video games, and the world wide web. Other important applications include accounting, airline reservations, avionics, banking, and telephony.
Software engineering is the community of practitioners who create programs. In the year 2000, this community included about 640,000 people in the U.S. and many more around the world. Many have computer science degrees, though some have science or engineering degrees, some have business or other non-technical degrees, and some have no degrees. They use diverse technologies and practices and work on diverse applications.
Software engineering has been very successful:
- Economic: Software engineers created a part of the growth of the U.S. and world economies during the last half of the twentieth century. Software engineers drove 1/4 of all economic growth (increase in GDP) during 1990s in the U.S., about $90 billion per year. Software drove 1/6 of all productivity growth (efficiency within GDP) during the late 1990s in the U.S., about $33 billion per year.
- Social: Software engineering has improved U.S. and world culture. Email, the world-wide web, and instant messaging enable people to interact in whole new ways. Software lowers the cost of many important social goods and services, such as healthcare and fire departments.
|
Directions for Software Engineering in 2003
Agile Processes: Agile processes are emerging as the most important new practice. Agile processes guide development month-to-month, week-to-week, and day-to-day. This allows projects to evolve as expectations and requirements evolve. The older document-driven processes (like CMM and ISO-9000) are fading in importance, partly because companies have exported many of the jobs that are controlled by these processes.
Aspect Programming: Aspects are emerging as the most important new programming technology. Aspects help programmers deal with ilities, because aspects describe how all objects or functions should behave in a particular circumstance. For example, aspects can add debugging, logging, or locking control into all objects of a particular type. Adding boilerplate code to many different classes would otherwise be error-prone.
Software Engineering Versus Programming
Software engineers work on all sizes of applications: small and large.
Programming emphasizes coding to produce working software applications, independent of budget and schedule.
Software engineering tries in addition to programming to encompass software projects more completely, including budget and schedule. Software engineering recognizes that software development fits in a large business context with relationships to marketing, sales, production, installation, training, support, and operations. Software engineering emphasizes methods to construct large applications that individual programmers cannot write alone. Software engineering tries to come up with methods to write programs in a consistent way.
Software Engineering versus Computer Science
Whether software development is more like science or engineering has been fiercely debated for many decades. Software development shares attributes of both and many software projects have elements of both, but distinctions exist.
Metaphorically, software engineering is to computer science and information science as traditional engineering is to physics and chemistry.
While many (most?) software engineers earn computer science degrees, they do not practice computer science every day, which is different than practicing software engineering every day.
| Issue | Software Engineering | Computer Science |
|---|---|---|
| Ideal | Useful | Correct |
| Goals | Working programs | Algorithms (like Shell sort) and abstract problems (like Travelling Salesman) |
| Budgets and Schedules | Software engineering problems (i.e. the next office suite upgrade) have fixed budgets and schedules. | Computer scientists problems (i.e. algorithms for NP) are independent of budget and schedule. |
| Emphasis | Real-World Problems: Software engineering emphasizes applying skills and working programs that deliver value to users. | Eternal Truths: The theory of software concerns the running time analysis, space analysis, and correctness of algorithms. |
| Change | Programs that will evolve as user expectations evolve. | When computer science problems are solved, the solution will never change. |
| Additional Skills | Domain knowledge | Mathematics |
| Notable Educators and Researchers | Barry Boehm, David Parnas, and Fred Brooks | Edsger Dijkstra, Donald Knuth, Robert Tarjan, Peter Slater[?], and Alan Turing |
| Notable Practitioners | John Backus, Dan Bricklin, Tim Berners-Lee, Linus Torvalds, Richard Stallman | Not applicable |
| Number of Practitioners in U.S. | 640,000 | 25,000 |
| Number of Practitioners in World | unknown | unknown |
Software Engineering versus Other Engineering Disciplines
Software engineers borrow many metaphors and techniques from other engineering disciplines. Software engineers as well aspire to build low-cost, reliable, safe applications. For example, software engineers borrow requirements analysis, quality control, and project management techniques.
Traditional engineers also borrow many metaphors and techniques from software engineers. Today, many traditional engineers use software tools to design and analyze systems, such as bridges and buildings. These new kinds of design and analysis resemble programs in many respects, because they exist as electronic documents and they go through analysis, design, implementation, and testing phases, just like software.
| Issue | Software Engineering | Traditional Engineering |
|---|---|---|
| Foundations | Software engineering is based on computer science, information science, and discrete math. | Traditional engineering is based on physics, chemistry, and calculus. |
| Cost | Compilers and computers are now cheap, so software engineering and consulting often cost more than 50% of a project. Software engineering cost overruns are large and easily affect project budget. | Construction and manufactoring costs are high, so traditional engineering may only cost 15% of a project. Engineering cost overruns are small and may not affect project budget. |
| Management Status | Few software engineers manage anyone, so they are not viewed as managers, except by themselves. | Many traditional engineers manage construction, manufacturing, or maintenance crews, so they are all treated as managers. |
| Innovation | Software engineers apply new and untested elements in many software projects. | Traditional engineers apply known and tested principles, and the amount of untested innovation that goes into each product is intentionally limited. Though some projects innovate. |
| Replication | Replication is trivial, and most development effort goes into building new (unproven) or changing old designs and features. | Most development effort goes into replicating proven designs. |
| Number of Practioners in U.S. in 2000 | 640,000 | 1,100,000 total engineers 65,000 computer engineers |
There are 10 times as many software engineers as computer engineers, so computer engineers are irrelevant to the software engineering community.
Software engineering is much younger than civil engineering and electronic engineering and much older than space engineering[?] and biological engineering.
Software engineering is already as predicatable and reliable as many fields of engineering, such as space engineering, biological engineering. Although large, reliable software systems can be and are constructed, software projects that fail during construction or in service are still too common. However, large traditional engineering systems that fail, like Three Mile Island, Chernobyl, Bhopal, Space Shuttle Challenger, are also too common.
Software engineering arose out of the so called software crisis of the 1960s, 1970s, and 1980s, when many software projects had bad endings. Many software projects ran way over budget and schedule. Some projects caused property damage. A few projects caused loss of life. As software becomes more pervasive, we increasingly recognize the need for better software. The software crisis was originally defined in terms of productivity, but evolved to emphasize quality.
Time and Money: One classic example is the OS 360 operating system, still used on the IBM 360 Series and its descendants. This decade-long project eventually produced a working system, which is amongst the most complex software systems ever designed. OS 360 was one of the first very large (1000 programmer) software projects. IBM eventually switched to Linux in the late 1990s, partly because OS 360 never lived up to expectations. Fred Brooks admits in Mythical Man Month[?] that he made a multi-million dollar mistake, when he managed the project.
Property Damage: Software bugs can cause property damage. Poor software security allows hackers to steal identities which costs time and money and hassle. The explosion of a European Ariane rocket and other disasters spur further developments in the field.
Life and Death: Software bugs can kill. Several embedded systems used in radiotherapy machines failed so catastrophically that they administered lethal doses of radiation to patients.
Silver Bullets and No Silver Bullets
For decades, solving the software crisis was paramount. Seemingly, every new technology and practice from the 1970s to the 1990s was trumpeted as a silver bullet to solve the software crisis.
Technologies: Every new technology and practice for decades was touted as the solution to the software crisis: structured programming, object-oriented programming, process, CMM, UML, Ada, methodologies, and so on.
Practices: Some pundits argued that the software crisis was due to the lack of discipline[?] of programmers. Many practitioners resist process strongly. Some believed that if formal engineering methodologies could be applied to software development, the production of software would become as predictable an industry as other branches of engineering. This led to work on code of ethics, professionalism. The issue is so pervasive that it cannot be deliberate. It must be a natural response to typical experience.
In 1987, Brooks published the famous paper No Silver Bullet[?], arguing that no individual technology or practice can make a 10 fold improvement in productivity in 10 years. All of the technologies and practices that practitioners use today make incremental improvements. Up to now he has been proven right, and he looks to remain correct for the foreseeable future. [[1] (http://www.virtualschool.edu/mon/SoftwareEngineering/BrooksNoSilverBullet)]
Women's Issues: In the 1940s, 1950s, and 1960s, software was a women's ghetto. Men preferred the higher prestige hardware engineering roles. So, women filled the lower prestige programming roles. Women like Grace Hopper were common. Many unsung women wrote code prior to 1968. Today, few women work in software engineering. Women have largely moved into analysis and testing roles. Saying that this is sexual discrimination is too simple, because it related directly to individual identity. In this sense, software engineering is the masculization of programming.
NATO: The NATO Science Committee sponsored two conferences on software engineering in 1968 (Garmisch, Germany) and 1969, which gave the field its initial boost. Many consider these conferences to be the start of the field. However, there has been a continuity of practice between the early projects from the 1940s to the 2000s.
Cost: The cost of software versus hardware has changed substantially, from the days of the mainframes, to the days of the PC. When mainframes were expensive, software projects could be expensive. Because powerful PCs are cheap, software costs must become cheaper, in comparison.
Criticisms of Software Engineering
Critics charge that the assumptions that underlie software engineering are inherently flawed.
Managing Expectations: Managing the customer's expectations to something that can be built and delivered is the key to successful "software engineering" methodologies. Thus, the field resembles marketing, sociology, or voodoo, more than traditional engineering with its responsibilities to society at large and the perils of legal liability with failure to protect the public interest. — Every profession manages expectations.
Poor Requirements: The requirements for most projects are incomplete and/or inconsistent. Some customers are first-time customers with little previous experience in writing requirements for a software project. Other customers really do not know what they want, and say "I'll know it when I see it" (IKIWISI). Even experienced customers who know exactly what they want may be unable to articulate their requirements. And, complete requirements may describe programs that have no computable or practical solution. Embedded systems are unique in that they are designed by other engineers, who can sometimes define interfaces completely. Users almost always expect far more than they write in the specifications. — One response is to not even try. Another response is rapid prototyping.
Rising Complexity: Critics argue that the probability of failure increases as project size, scope and complexity increase. Technologies and practices have improved through the 1970s, 1980s, and 1990s, but the complexity in requirements and user expectations increases fast enough to overwhelm the improvement in technologies and practices. So, the gap between what is expected and what is delivered has not improved. — This is not the fault of practitioners. This is actually a measure of the success of practitioners.
Ongoing Change: Practitioners are continually eager to develop new technologies and practices and try new tools in every project. Some view this ongoing change as proof that older technologies and practices were failures. — Others view this ongoing change as proof that the field successfully learns and grows.
Ongoing Failure: Critics charge that incomplete or poorly designed systems are still too common. The early lessons of the field were not sufficient to prevent subsequent disasters. — Yet, no field that strives to do bigger and better projects has ever avoided all failures.
Nothing New: Critics argue that software engineering has not created anything on its own, it merely uses what computer scientists already know. This is more-or-less true. — But, practitioners developed many tools and practices (compilers, make, cvs, XP), on their own, out of simple need.
Anyone Can Do It: Many bright people from other fields (engineers, scientists, business people) write spreadsheet templates or calculator programs, and eventually switch to writing large applications, and believe that they do software engineering. And many of them are. — Software engineering is a skill that is refined through long practice. Software engineers are the ones who already have the education and experience, and they keep up. This is true of any skill.
We Do Not Know What It Is: This is often said by people who want to redefine software engineering on their own terms. — Software engineering is the technologies and practices used to create office suites, databases, and engine controllers.
Software engineering methodologies span many disciplines, including project management, analysis, specification, design, coding, testing, and quality assurance. All of the methodologies guiding this field are collations of all of these fields.
Software design methodologies can be informally classified into thick and thin. Thick methodologies include a large amount of formal process paperwork and documentation.
The well-known thick methodologies include Cleanroom[?] and Rational Unified Process (RUP). Thin methodologies eschew formal process paperwork and documentation. The well-known thin methodologies include Extreme Programming (XP) and Agile Processes.
Recently, some (like Karl Weigers) have argued for no more methodologies. Methodologies tend to list the contemporary technologies and practices and insist that everyone use them. This advice is obvious for those who have the opportunity to use contemporary technologies and practices, and useless for those who maintain legacy systems and must use legacy tools cannot do so, due to circumstance. So, methodologies are not specfically useful. And methodologies must be updated as technologies and practices evolve.
There is a growing body of software development organisations that are implementing process methodoliges. This is mostly driven by the defence industry that are requiring a 'Rating' baised on 'Process models' as a part of their contracts.
For example the Capability Maturity Model (CMM) describes the experience level on which a software creation organization works. ISO 9000 is a standard of describing and organizing the process with the documentation in a formalized way.
ISO 15504, The software process life cycle is also gaining wide usage. This standard is aimed at setting out a clear model for process comparison. "SPICE" as it is known is used in a similar manner to CMM or the new CMMi. That is as a model of a set of processes that can be used to manage, control, guide and monitor software development. This model is then used for comparison against what a development organization or project team actually do during software development. This information is analyzed to identify weaknesses to drive improvement, and strengths so that they can be continued or integrated into common practice for that organization or team.
Different methodologies advocate conflicting solutions. Proponents of different methodologies often get into heated debates over their merits.
Many books have been written about software engineering. The following books are milestones by ex-IBM people.
- Fred Brooks wrote the The Mythical Man-Month, at UNC describing what he learned from the OS 360 project.
- Watts Humphrey wrote The Capability Maturity Model[?] for the Software Engineering Institute, emphasizing management.
- Gerald Weinberg wrote The Psychology of Computer Programming[?] as an independent consultant.
Professional Software Engineering
In the mid-1990s, The National Society of Professional Engineers sued in all states to prohibit anyone from using the term software engineer as a noun or field of employment. They won in most states. Utah does not license professional engineers, so the point was moot there.
In response, the IEEE and ACM began a joint effort called JCESEP[?] in 1993, which evolved into SWECC[?] in 1998 to explore making software engineering into a profession. The ACM pulled out of SWECC in [1998?] objecting to its support for the Texas professionalization efforts. The IEEE continued to support making software engineering a branch of traditional engineering.
Donald Bagart of Texas became the first professional software engineer in the U.S. on September 4, 1998 or October 9, 1998. As of May 2002, Texas had issued 44 professional enginering licenses for software engineers.
The professional movement has been criticized for many reasons. The professional movement uses the traditional engineering model, so licensed software engineers must eventually learn years of physics and chemistry, which is irrelevant to most practitioners. It ignores the question of what to do with all of the computer science majors, who don't earn degrees in engineering schools.
Another criticism of the Canadian effort is that most people who become professional software engineers actually studied computer or electrical engineering, even though these people already qualified to become professional engineers in their own fields. Thus, professionalism is simply another tool for traditional engineers to prove that they are better than computer scientists and software engineers. It totally failed to improve software engineering.
- http://www.faqs.org/faqs/engineering/pe-eit-exam/ Apr 10, 2002
- http://www.sdmagazine.com/documents/s=746/sdm0003g/0003g.htm
The fields of data engineering, knowledge engineering, and so on have similar concerns over engineering issues.
Software Engineering Technologies and Practices
Technologies and Practices
- Programming languages, like Java and C++. Also includes compilers, linkers, and loaders.
- Relational databases, like SQL.
- Software tools[?], like make, cvs, editors.
- UML, design language
- Patterns and Anti-patterns document many common programming and project management techniques.
- Software development processes, like RUP, XP, Agile programming.
- Platforms, like WINTEL PCs and IBM mainframes. As platforms have become more powerful, tools have become available on less expensive platforms and more widely available.
- Software metrics
- Several approaches to the programming in practice:
Life Cycle (Waterfall Point of View)
- Project lifecycle
- Requirements gathering and Analysis
- Software architecture
- Functional decomposition
- Computer programming
- Testing, focuses on detecting (preventing?) bugs
- Quality assurance, ensures compliance with process, but it is a misnomer, it is not quality and not testing
Management (Management Point of View)
- Project management
- Configuration management
- Personnel management[?]
- Risk management, everything is a risk because everything could cause delays.
Notable Problems of Software Engineering
- John Backus: Fortran, first optimizing compiler
- Donald Bagert[?]: First professional software engineer in U.S. in 1998.
- Kent Beck: Refactoring, XP (pair programming, test driven development).
- Barry Boehm[?]: Economics, COCOMO, spiral model.
- Grady Booch[?]: Object-oriented design, UML.
- Fred Brooks: System 360, OS 360, Mythical Man Month.
- Michael Fagan[?]: Code reviews.
- Grace Hopper: First compiler (Mark 1), COBOL.
- Watts S. Humphrey[?]: CMM, headed (founded?) the SEI.
- Jean Ichbiah[?]: Ada
- Bill Joy: Berkeley Unix, VI, Java.
- Brian Kernighan: C and Unix.
- Peter G. Neumann[?]: Computer Risks, ACM Sigsoft.
- David Parnas[?]: Module design, social responsibility.
- Dennis Ritchie: C and Unix.
- Winston W. Royce[?]: Waterfall model.
- Will Tracz[?]: Reuse, ACM Software Engineering Notes.
- Richard Stallman: Founder of the Free Software Foundation
- Gerald Weinberg[?]: The Psychology of Computer Programming.
- Ed Yourdon[?]: Structured programming, The Decline and Fall of the American Programmer.
- Bill Gates: Microsoft
- Larry Elison[?]: Oracle
- Mitch Kapor: Lotus (now with the Open Source Applications Foundation)
- Compilers
- Computer graphics image processing and special effects for video and film
- Embedded systems
- Graphical user interfaces
- Office suites
- Operating systems
- Relational databases
- Video games
- World wide web (Tim Berners-Lee)
See also computer science, programming paradigm, information engineering[?], knowledge engineering[?], SWEBOK.
For more information on the software engineering community, see:
- http://www.software-engineer.org
- http://www.agilealliance.org
- http://www.acm.org The Association for Computing Machinery
- http://www.computer.org The IEEE Computer Society
- http://standards.ieee.org/software/index IEEE Software Engineering Standards
Magic: If the programmers are good, the system will work, and the customer will not fully appreciate how difficult the task was.
Maintenance: Most (70% or more) software engineering effort over the total lifetime of a system goes into maintenance and upgrades.
Delivery: In the course of taking a large software project from conception to end user acceptance (and actual use) the cost of developing the software will typically range from 20-30% of the total. Other activities (documentation, Training infrastructure , Support infrastructure, Deployment and Network design, etc) account for the other 70-80%.
We all agree that we want better software. We disagree on priorities and approach, on what an individual should do first in a specific circumstance.
Get the number of software engineers in the whole world for 2000 (including the source)
Competition and time eventually weed out bad ideas, assuming effective competition exists.
Expand the software crisis examples.
Some examples here of successful large projects, where software engineering methods have been applied, would be nice. (In the Software Engineering Versus Programming section.)
All Wikipedia text is available under the terms of the GNU Free Documentation License
|
Search Encyclopedia
|
Featured Article
|