|
The benefit of the 'smoke test' |
 |
 |
 |
|
Written by Jeb Bolding
|
|
Friday, 20 August 2010 23:13 |
|
Testing is one of the phases in product development that tends to suffer when schedules aren't being met by engineering. As the engineering schedule expands beyond the original delivery date, the testing window shrinks. It's a sad truth that has no positive consequence. It's also particularly rampant in startups where a QA department is usually an afterthought. So, as a product manager, how do you cope? I believe that's where the smoke test comes in. Simply stated, anytime you have product requirements, you have some basic use cases that drive the thematic workflow for the features in the project. In the case of a smoke test, you turn those use cases into the basis for product testing. Use cases become test cases. Smoke tests are valuable in several ways. From the product perspective, a smoke test gives you a sense of how well the key user workflows will actually work. One problem with becoming a serious tester is that you can get lost in the weeds of development and bugs. With smoke tests, you lift your head above the trees to gauge the status of your forest and the real needs of your users. Another positive for smoke testing is reporting up the chain to execs. Most aren't interested in the details of what's working and what's not working, but, if you say, "there are five things that our users need to accomplish with these sets of features and two are blocked by bugs," that's understandable as 3/5ths of the product works and easy to dive into for an understanding of where the project is. I tend to structure the smoke test very simply. I like spreadsheets for this. I create the key use cases in columns and include the five or six pieces of the use case that are important. I tend not to have more than six or seven steps in each column. Next to that column, I have a date and and empty column. For that date, I go down the list and highlight with red or green whether or not that step works. If the process is blocked at one of the steps, I highlight that step as red and all the subsequent steps as red. A few days later, and, on a regular basis, I go back and add another column next to the previous dated red/green column and test the use case again, filling in red and green blocks. Hopefully, over time, the number of red blocks in the new column will be fewer than the red blocks in the previous week's column. |
|
Last Updated ( Wednesday, 01 September 2010 22:31 )
|
|
|
Written by Jeb Bolding
|
|
Friday, 10 October 2008 22:44 |
|
I often go by the451 Open Source forume (CAOS Theory). Noticed an interesting position about open source adoption impact in a down economy. http://blogs.the451group.com/opensource/2008/10/08/the-other-value-of-open-source-in-this-economy/ Instead of retyping or copying my comment, check out the above link. Well, one link wasn't enough. I also just had to throw in my 4.75 cents worth around an OS X based Netbook. http://blogs.the451group.com/opensource/2008/10/02/apple-may-mix-up-netbooks-linux-still-looks-good/#comment-283712 |
|
Last Updated ( Friday, 10 October 2008 22:50 )
|
|
Written by Jeb Bolding
|
|
Thursday, 11 September 2008 00:00 |
|
Timing One of the things that product development books really never talk about is timing. Timing the product, marketing or idea that you're proposing. There are plenty of great ideas that make it into the market...and, fail; only to be resurrected, in seemingly identical form, a few years later; all to great fanfare and success. Why? First, note that I don't believe there is such a thing as definitive metrics that will tell you if your timing is correct...if there were, then we would all use them and every product would be a success (minus the impact of execution). Though, that's in parentheses, I don't want to downplay the impact of product/market execution. As one VC said to me once, I'd rather invest in a mediocre idea with spot on execution than a briliant idea run by flakes. Since there are only directional indicators that help in determining timing as a butress to the release of your product, what are those indicators and how do you determine that those indicators are present in a given market? Indicator 1 Product consumers believe that the problem is worth money to solve, and, are willing to pay more than a $ for it.  Indicator 2 Corollary to the first indicator, the general financial market supports the product as either a boom product or contrarian product. Indicator 3 Your product doesn't require explanation to the majority of survey-ees, the "aha" moment is quite obvious to the potential consumers. {it should be noted that this is all about timing...I'm not presuming that some products can come out before the market is ready, it's just that they have missed market timing and may require significant evangelization.} Indicator 4 If you are in an existing market, you have a way of surpassing the value of competitive commodities. And, if you believe you're in a blue ocean market see indicators 1 and 2. This is one of those situations where I could probably add another five indicators. However, these are the ones that if any of the market research comes back negative on, you have an uphill battle and will spend most of your time evangelizing the need and usefulness of your product. All that said, once you have the indicators down, how to you go about validating a yea! or nay! to the indicators? Indicator 1 Validation |
|
Last Updated ( Wednesday, 24 September 2008 22:43 )
|
|
|
Surrounded By Subject Matter Experts |
|
|
|
|
Written by Jeb Bolding
|
|
Wednesday, 08 October 2008 16:19 |
How to Make Sense of SME AssumptionsI've walked into several companies with little or no expertise in the market that the company addresses. That being the case, I'm almost always faced with the question of how to become useful in a relatively quick amount of time. These people know more about their customers, know more about their business, and have the internal personal relationships in place that allow them to get things done within the organization. All of which, I don't have. I tend to have a fallback position walking in as the new product manager. My first meetings in the company tend to focus around features and customer need perceptions around those needs. I build a feature prioritization weighting with the goal of adding methodological introspection around WHY the features are important and what impact they will have for the company and its customers. The next step (well, some might argue that it's really the first step), is to create a series of personae which model the end users as well as the channel users (if there are any). These two steps are important because in both cases, you are asking the SMEs within the company to think about their customers, and their customer's needs in a systematic way. Why? It's often the case, I've found, that companies, particularly small ones, are so involved in their customer development as they evangelize their products, that their assumptions about their users have become truisms that few tend to question. Often, we call this "drinking the Kool-Aid." It's not as if creating a spreadsheet of features and weights is hard, it's not, nor is creating a few personae with attributes. But, most companies, large and small, don't regularly evaluate their assumptions about their customers; certainly, not every six months (1/4 of a lifetime if you have a web business). Coming up with key features, evaluating them against personnae and business objectives and then discussing priorities with vested interests will do two things for you. First, it will make you immediately valuable because it's inevitable that people will have "aha" moments looking at your documents and think that you're really smart. And, second, it will be the first step in transferring their SME knowledge to you. The next question, then, is how you keep yourself from drinking the Kool-Aid and getting lazy about your product and customer assumptions. |
|
Last Updated ( Wednesday, 08 October 2008 17:13 )
|
|
|
