“It Starts with an Idea” is finally published and available on Amazon – Click here.
Read about the adventure of a start-up in Silicon Valley. The book interlaces the story of the start-up Intelic (renamed “Azerity” in 2001) with stories from prior jobs, valuable lessons learned. The book reveals how Intelic created a dedicated and enthusiastic team – a team that worked hard and had fun – and a great product.
Draft copy of “Practical Software” being reviewed! (UPDATE: We changed the name to “It Starts With an Idea” before publication)
Some teams seem to have a very hard time managing their bug process.
Even teams who implement good processes sometimes have setbacks when budgets get tight and trade-offs are made. In large and small companies, often management pushes for more features so sales has “new stuff” to sell and bug fixes go by the wayside. Sometimes the code itself is not good due to a rush to get something out-the-door. Teams often end up with a huge technical debt: piles of bugs that need fixing; code that is brittle and hard to maintain.
The team I’ve worked with for 15 years has implemented great processes and tools to stay ahead of the game. Through the years we’ve learned what works and what doesn’t. Here’s a list of do’s and don’ts to stay ahead of the “Technical Debt” debacle.
Avoidance is the best policy. It’s really not that hard. Here’s what you need to do:
What if you find yourself in a group with significant technical debt? If you’ve decided you need to get rid of your huge pile of bugs, you may find yourself spending inordinate amounts of time and effort sifting through the bug calls, listening to client complaints and trying to prioritize which bugs to fix first.
Instead, start with a process and stick to it. The best way to get out of your hole is to stop digging.
First you’ll need to decide if you need any major re-architecture projects. If you’re an IT team, you may need to look for a vendor solution to replace your in-house tools. Look for one from a company with good processes and clean software.
If the problem is a huge pile of unaddressed bugs and customer issues, here’s a good approach:
Priority and Severity are normally not coupled. Normally a Severity = Critical will also be Priority = High unless, perhaps, the data degradation isn’t occurring because the client hasn’t turned on some related function. Otherwise there’s little linkage. Even a Severity NonCritical bug could be Priority “High” if it is a misspelling that makes the company look bad, for example.
Assign to upcoming releases based on a combination of Severity and Priority. Lower Severity and Priority items can get put into a maintenance bucket to be worked off after the more important ones are done.
Why? Because if these are not assigned somewhere, the team continues to thrash through them, again and again. If they get assigned, even if some adjustments are needed, we have a sense of when fixes can be made available for the user.
Instead of continuing to look at them and thrash through them again and again, they are ready for developers.
It’s best if you have a good bug tracking tool that allowed you to change assignments and ask questions, capture all comments, and record history. At my company we used the in-house tool SD Tracker. That tool was the impetus for building Software 2020.
Regardless of what tool you use, everyone needs to use it and it needs to be easy and effective. It can be done with spreadsheets – it’s just a lot harder.
I’m back to blogging now having returned from our great 6-week trip on our new boat in Canada.
I have been contributing to various Agile LinkedIn and other forums – about what works in Agile, what doesn’t. Whether or not Scrum = Agile, if you can offshore and be Agile, how to develop with agility, etc., etc.
The funny thing is that what I blog about is more often my experiences with our “Practical Software Methodology”, not “Agile”. From what I see in blogs and discussion groups, “Agile” (with a capital “A”) means Agile/Scrum or Agile/Kanban – strict adherence to every step and recommendation like a religion. Whereas there are a lot of processes that can follow the Agile Manifesto’s tenants and principles and not be Scrum or Kanban. I call those “agile” (little “a”) or Agile (without the quotes). I think they are realistic for the majority of today’s software projects.
“Practical Software” was the process implemented at Azerity, my start-up. For years prior to starting Azerity, my goal was to take the best practices learned at Ford Aerospace on Government projects plus leverage the work we did with Carnegie Mellon on SEI Assessments (now referred to as CMMI) but throw out the bulk resulting in a lean and clean process for start-ups. As my start-up company grew I was able to hire a top manager from Ford Aerospace and together we implemented the tools and processes we later coined “Practical Software”.
With our Practical Software Methodology we were iterative, produced code quickly, short release cycles, high quality and high customer acceptance. In parallel, we maintained our on-line DOORs requirements specs, our “bible”, which were part of our process and which were accurate, tested, used by developers, QA, the Call Center, and Consulting. When we sold the company, the on-line DOORs specs were key to the value of the IP. Other developers could be hired, people come and go and even I could quit and the DOORs specs lived on. We were able to maintain those accurate documents with minimal cost because they were part of our development cycle and were iterative. They were not big waterfall Word docs but living breathing statements which matched the tasks assigned to developers to implement/update.
We followed the Agile Manifesto tenants:
Agile and it’s related methodologies (XP, Scrum) were born in parallel to Practical Software. I was previously a skeptic about Agile because whenever I heard someone talk about implementing “Agile”, they also added “throw out documentation”. Just talk. Sticky notes on a white board. Since my background is building large-scale mission critical software applications with extensive business practices and policies incorporated in the code, where the product was already so large even I was having trouble remembering every business requirement, I couldn’t envision how our product would/could ever be managed, evolved, and maintained. Plus our process which included on-line documentation was so clean and efficient it was hard to understand why other companies were struggling. (I’ve since come to realize that others haven’t found the secret sauce – how to maintain documentation in parallel without impacting development timelines).
A few years ago I was converted from an Agile skeptic to a proponent. The company that bought Azerity was implementing Agile or a hybrid thereof. I became a certified Product Owner. The interesting thing about my Agile training was that my objections to Agile ended up being objections to some form of Scrum or to a misunderstanding others had of the Agile principles or to some other misuse of Agile tenants. In reality, what I learned was that at the core of Agile (the Manifesto) were the very concepts that were at the core of Practical Software. Which makes sense. The initial proponents of Agile were seasoned software professionals who looked at how waterfall/phase-gate projects were being done, saw the issues, and proposed a better way. We at Azerity looked at how the big government waterfall projects were done, saw the issues, and proposed a better way. Many if not most of our findings were consistent.
I now believe the industry is being improved overall by the consistent message and training coming out because of Agile. I still see issues in the Agile communities:
Both Agile and Practical Software solved what I saw in government and commercial software companies as the main issue – the issue of organizational silos and phase-gate workflow. “Business people and developers must work together daily throughout the project.”
I am now an Agile proponent (or should I say “agile” proponent). Please excuse me if I’m sometimes blogging about my “Practical Software” experiences ☺
To better describe how Software 2020 can help you create your As-Built documents as a by-product of your software development process, automatically, here’s how we used it when we built Software 2020. Of course we couldn’t use it from start-to-finish like we recommend you use it because it wasn’t built yet ☺ But we proved how great a tool it is and will continue to use it as the product evolves.
I first wrote a spec to capture my thoughts and ideas since there was no Software 2020 to put my ideas in but we used it as a guide, not as firm requirements. Freeman is so creative that when he doesn’t need to follow a spec exactly we get better products. Similar to why Agile proposes fuzzy stories which evolve into the final firm specification. We work best as a team when we can iterate and evolve the product as we go along. We’re agile.
So we started fuzzy (like you would with Agile stories). I want a “Task Tracker” tool like our prior SD Tracker (proprietary) tool. I want another document-type, Word-like “view” of the subset of tasks that were requirements or “specs”.
I’ve built enterprise application software for years and managed those efforts, using a combination of tools and proven processes that allowed us to build products quicker, better, and with less people than other software companies could in our field. Our techniques and approaches have withstood the test of time. I knew how I’d like one tool to solve the holes and issues I’m seeing in how companies use tools today: Agile Story Managers, Task Trackers, Requirements Management Tools – and definitely better than what is used in Agile teams that don’t like tools: Word, Excel, Story Boards. I had a good idea of what I wanted. Freeman had great ideas about the new kind of architecture he wanted to build.
The Tracker module was completed first since we had a clear starting place, improving the SD Tracker tool we built at our prior company. We started our project in October and by December the Tracker module was functional enough to use it to enter all of the development tasks in. I took the list of spec/tasks we’d been working from and entered them in the Tracker and from then on we worked as we always have in the past: Nothing gets coded unless there’s a Tracker Task for it and the task is assigned “open-fix” for development, set “in-progress” when work begins, “resolved” when on our integration system ready to test and “closed” when I’ve tested it.
Since by this point were already screens built for the Tracker, we started working to make sure those matched my ‘ideas’ and had some minor corrections. For example, the first task I entered in the Tracker, #1001, was a clean-up task:
As we did the clean-up, we evolved this task to capture the final requirement for screens in the Description of the task:
“There are three types of screens: Summary Screens that present batched sets of information, Detail Screens (Task, Person, Client), and Pop-Ups (attach a New Attachment, Set Defaults, etc.)
“On all screens, in the white top banner, there’s the Software 2020 log on the left and Navigation (Nav) buttons on the right. The first Nav link goes to the user’s Profile. The user’s name should be shown as ” Profile” in the tab bar. That way tech support knows who the user is but the screen is cleaner.
“Below the white banner is a blue Title Bar. The Title should be white and centered in the blue title bar.”
Once we had an “Organize View” feature in February 2012, I selected tasks that represented requirements, like the one above, #1001, and linked them to the appropriate spec until all tasks that represented requirements were in organized specs, by module. Here’s the current Software 2020 Spec List:
Or closer up view:
When I drill into the Look & Feel spec I see this:
Note that it looks like a nice word document (embedded graphics, etc.). That first task, #1001 is there. If you look at the entire screen you can view task information to the right so it’s not just a Word Doc but is a nice way to view, and update spec tasks:
Now I have all of the information about what was built, each and every task, in a nice readable set of specs. This is so much easier and better than trying to maintain a separate Word document or even update requirements in a linked tool. And definitely an advantage over having no documentation at the end except your pile of yellow stickies and code, regardless of how well commented and structured the code is.
Some advocate the Test Driven Development (TDD) approach. I agree there is benefit to TDD and would like to have a Test View on the same set of completed tasks to create test plans from. But disagree that TDD alone would replace real business-oriented requirements specs.
There’s a “Download” button if you want to save versions as Word docs, Excel spreadsheets, or an HTML page that you could store on a central server after each release to have a set of documents reflecting exactly how that version was built.
Then, as the Product Owner, I can add new features for the next release as Stories or requirements right in the Organize View (which is how I like to do it because it’s organized and logical, like a spec or word document) and they show up as tasks in the tracker. I could also, for new features entirely, create a Story Spec and put Themes / Epics / and Stories there.
New spec tasks and any other tasks (bug fixes, maintenance tasks, etc.) assigned to a release can have Story Points or time estimates entered and can be easily prioritized using the Ranking Tool:
Once ranked, the team can assign to Sprints and there’s no re-entering of tasks needed, no extra work.
Prior to Software 2020, we spent 10 years developing robust, secure enterprise application software using the prior SD Tracker tool and the Teleogic/IBM/DOORs Requirements Management System. Our software competed with the big guys – Siebel, Salesforce, SAP, Oracle – and won. Our process and toolset worked effectively but had manual steps and we didn’t have Agile story management tools.
Using Software 2020, the “One Tool” to do it all, is so much cleaner, easier, more efficient, and because of the automatic linkage, clearer for the developers than having to use a tracker then open a requirements spec, look for the links, etc. The task has the link to the spec – one click and they see this task in context with other related information (tasks related to the same function, screen shots, etc.)
For Agile teams, for the first project you would enter Stories grouped under Themes and Epics in the Organize View. Software 2020 provides nice Agile story management tools (charts, ranking tool, story point calculator) to help organize the sprints. As the story evolves, if the PO simply keeps the task/story description up-to-date, and, because it IS the task the developer is working on, there’s no mis-understanding. Because that description is on the task the developer is working on, both the PO and developer can easily view/agree that the description does match the code. There isn’t a separate “Requirements Spec” to maintain. Nothing to get out-of-sync. It’s automatic.
With Software 2020, the company obtains the benefits of having good systems documentation without any added work.
See how our new tool, Software 2020, can produce automatic documentation as part of the project.
(Copied from Software 2020 Blog)
On a recent LinkedIn post, people were saying “we still struggle … when it comes to testing and documentation” and not having good design documentation “creates difficulty for the software testers to come up with tests …” and that “In regulated environments, documentation is a must … and most important tracing from requirements to design and tests.”
I think you can have both. I believe having traceability from the design specification to tests and ending up with an accurate “As-Built” Spec is a “must” and valuable even in Agile environments. What “isn’t” Agile is expecting everything to be completely designed up-front. Rather I believe in evolving the requirements (or fuzzy up-front stories) but at some point the task must become clear enough for implementing.
My friend and I were VP Engineering and CTO respectively at my software company. She and I had both come out of the Aerospace industry and wanted to take what we’d learned and throw out the unnecessary steps to create a light-weight, high-quality process for start-ups. We called our process “Practical”. People who saw how we worked said we were agile. One key was we integrated an on-line requirements management tool tightly into our process. I’d update the new features in the on-line specs. Some would be complete up-front if the change was well-know and researched. Other times there’d be a placeholder – “We want a configurator”. Developers worked from these on-line specs. We were iterative – sometimes the specs evolved with the prototypes and the inventiveness of the developers but before testing started, the specs had to be finalized. Testers tested against the spec updates. I had a lot of manual work to link tasks to specs and make sure the wording was always up-to-date as the tasks evolved. But well worth it. The whole company leveraged those on-line specs. Extremely valuable.
But I always yearned for a better integrated tool where the story was automatically a task so everything was in one place. And as the task was split, refined it stayed organized, like in a spec so at the end I had accurate, on-line description of the actual deliverable product.
Companies now embracing Agile (completely or some hybrid) start with an Agile Story Manager tool and often need to re-enter tasks into their tracker. In companies that used to have good, integrated specs as part of their process like ours, those specs are becoming inaccurate, aren’t used by developers to code from and aren’t tested. Worthless.
We built Software 2020 because of these needs. It’s the tool I’d always wanted and we’ve been using it to build the tool itself (how’s that for iterative 🙂 There’s an “Organize” view that lets you structure your stories in a word-like doc (you could start with themes and epics but if you had an existing organized view “spec” you’d add the new features in their proper place) with screenshots, etc. and each specification is automatically a developer task. You can update the description of the specification as design is clarified either in the “Organize” view or regular task Tracker screens. Later we want to add another “view” for the test center. That gives the linkage the regulatory projects need – automatically. But even for non-regulatory, even on our little tool-building project, my architect/developer is already finding the value of looking at the document view of my design spec, in-context, to better understand the new features I’m designing/asking for. So I think we can be Agile and have nice documents too.
I posted an Agile LinkedIn discussion group question Where do you go to see what the code really does besides asking a developer to start a discussion about the role of specifications in the new Agile methodology. The discussion threads have been interesting and lively (LinkedIn, group “Agile”). So I wanted to answer the question posed numerous times: “Who would use that documentation and why/how?”
First, I’m not talking about old, antiquated waterfall design specs that are soon out-of-date used by no one. I’m talking about living, breathing hierarchical information (in a central tool accessible by everyone) that represent the repository of product knowledge: protected, validated, for use by the entire organization. Documents that everyone in the organization takes ownership in and uses. A cool set of “shiny docs”.
I can answer later how it could have been done (seems it’s not done at most companies and recent surveys say 83% of companies at best use Word Docs to describe their requirements) – but for argument’s sake, trust and believe there is a repository on-line specifications (“shiny docs”).
Who uses our current on-line specs and why do they need them?
a) For the developers – for the new guy, not as a replacement for training and not a static user manual – rather a handy reference area to delve into for more detailed, complex functional understanding, even some design details. A wiki for the product, helpful even for the most experienced developers.
b) Technical Support (TS) (the Call Center) wants to be able to quickly identify if a customer request is a bug, training issue, or if it should become a new feature request. They do their initial investigation using the specifications and replicate the issue in-house if they believe it’s a bug (so they help identify if it’s a customer config problem or base code). For non-bugs, they use them to understand how the product was designed versus what the customer is requesting to give better feedback to the PMs and better represent the customer’s business position for the change versus what’s there. Having a knowledgeable TS team that can get accurate information about all features in a very large enterprise application streamlines getting real bugs and issues into the product development releases. It’s much quicker if everyone knows/agrees a fix is needed than if something just sits on the backlog pile until the next PM release review cycle. Customer satisfaction is improved with clear feedback about if a ‘fix’ is feasible and how quickly. Some product changes can’t be immediate. Customers get that. Clear communication either way improves customer satisfaction.
c) The Product Management Team: If you are a Product Manager for a huge enterprise application and need to add new features and functions requested by customers to an older module with tons of business functionality, you probably don’t know everything about what the product does or even all of the history about what it should do and why customers wanted it that way. The online specifications help immensely.
If such documentation could be produced automatically as the result of the development process – wouldn’t it be useful? Why not aim for that?
(Posted to Duck Pond from Software 2020)
Aren’t new enhancement requests, stories, bugs, and requirements all just “Tasks”? And instead of all the different tools today — Agile Project Backlog Management Tools, Bug Tracker Tools, Requirements Management Tools — shouldn’t those be just different views to manage these (or subsets of these) “Tasks”? I think so.
Story Boards, Requirements Specs, Bug Tracking Tools are all just different views of “Tasks”.
Given that perspective, a modern robust Tracker would manage all “Tasks” (whether they are the new Agile Stories, bugs, client issues, etc.) consolidated for team tracking and estimation to complete reports considering the team’s vacation calendar and all of their work combined.
Then modern Agile tools could help plan new releases taking into account information about the developers’ other work (hence their percent availability) and provide a better estimate of completion for the current sprints and releases. Useful tools then would help manage the current release (in Agile, that’s Story Boards and organized hierarchical views – grouping tasks into epics, stories and their children). And facilitate the team’s management of these Tasks as they move from concept to delivery.
But keep them in the bigger database of “Tasks” so managers can manage the team’s full effort, not just the current release.
That’s what Software 2020 does.
What I worry about are where are the final documents that describe what the software does? That seems to get lost in Agile. And is sooooo important for long term maintenance, Technical Support/Call Center and years later Product Owner updates.
Today documentation is “Not Agile” and I question that. Before Agile if you had a real requirements management tool (we used DOORs) there were requirements, developers would look at the DOORs specs (linked to our internal task tracker for any new feature), during development they’d give feedback if they needed DOORs to be updated (it was fast, real-time, agile), QA would test against the tracker and DOORs and we then knew the DOORs specs accurately reflected what the code did. What a timesaver for Tech Support (the Call Center), PMs and others to know what’s in the code without having to go back to the developer. Without specs you seem to loose that.
Not the waterfall design docs Agile eschews that are in Word and get out of date before the code is delivered – but real information somewhere like in DOORs that says what the product is doing. Information the developers worked from and that QA says is how the code works.
So while I’m totally on-board with the Agile concepts for the software development itself, what is the answer for the rest of the company to know what’s in the code? Are we back to having to have the software developers go look at the code and tell them? Stories in tools like Rally are for the current release. And Agile doesn’t say how to update them or what happens for the next release. Agile “Stories” are incremental. After two or three releases, where does someone go look to see what the code does – other than asking developers? (which is a waste of their time).
That’s one of the big problems we’re hoping to solve or facilitate with Software 2020.
In the ’90s we found the perfect software process and the right toolset. Our process was practical and results amazing. Leveraging managers with years of software management expertise together with very talented practical software architects, the Azerity product demonstrated that software can be done right. Azerity competed with the big gorillas (SAP, Siebel, Oracle) and won every time. And it didn’t cost the clients millions to install and millions more to upgrade. High quality, intuitive. Some said it was nearly perfect software.
What did we learn?
Our management tools were SD Tracker and DOORs
Results
Others who saw our process said we were “Agile”
Duck Pond Software was created to provide support to software companies on architecture, product, and process. As the 2000s have progressed and more and more teams are moving to Agile, I’m finding that Agile is helping teams move to better software development methodologies and how the ’90s processes can be updated to help support Agile teams.
But I’m also finding the current Agile tools are lacking. That’s why we’ve started software2020.org and are developing Software 2020. A demo will be available soon. It combines the best of the Azerity processes and tools plus Agile. There’s more to come . . .
Jan McCleery - from the Duck Pond 