How I feel after trying most of my experiments (thanks to clement127 on Flickr)

Product Management Experiments and Outcomes — 2019

One of the salient lessons I learned early in my transition from GM to PM, was that language really matters. Some (meetings magically became “ceremonies”) matter less than others (there’s a big difference between safe and SAFe). My favorite example of this is the in the prioritization of learning.

My mentor and the team he built emphasized thin slicing, iteration, and experimentation all for the sake of learning. Each slice delivered into prod either taught us something by doing it (e.g. how to work with containers, how to run more thorough testing) or something about the production environment (e.g. is our instrumentation good enough, what is the latency we’re experiencing in production).

I was reminded of this last week when I was chatting with a colleague about a business strategy retro that we need to do. She commented that successfully understanding what went well, and what didn’t, will help us “fail fast”. I corrected her: it will help us “learn faster”.

With that in mind, I wanted to review the PM experiments our team ran in 2019, and what we have in mind for 2020.

2018 Experiments

Daily Huddle

• What it is: Contrary to what some may think, standups are awesome for tactical problem solving. Accordingly, our team experimented with getting rid of the weekly 1:1s between me (PM team lead) and each team member (PM for a Scrum Team) and instead do a daily 15 minute conversation with the entire team. The focus is to highlight what work you’ve got coming up, and what needs escalation/support. The invitees are all of the PMs, tech writers, and tech leads for our group (6 PMs, 2 Tech Leads, 1 tech writer).
• 2018 Result: Massive success. This ceremony allowed us to avoid the weekly tactical meetings, gives the broader team visibility to the work around them, allows them to self-organize for finding support, and allows me to share more frequent context on the work we’re doing. Additionally, this keeps our biweekly 1:1 feedback conversations focused on personal growth, rather than tactics.
• 2019 Updates: We added a virtual huddle on Fridays. See below for more details.

Sprint Update

• What it is: Every sprint, we send out an update on each project we’re working on, the R/Y/G status, and a feature map which shows how we’ll build the slices that get us to feature complete.
• 2018 Result: Controversial. This will be the subject of a separate post, but there’s good and bad here. Good: dramatic increase in transparency, tons of shareable material that prevents one-off meetings, helps promote the team and our culture. Bad: still requires active engagement from partners, and we haven’t yet cracked that nut.
• 2019 Updates: Better, but still controversial. The Sprint Update has continued to improve through better use of our Google Suite tools. Our tracking tools and transparency are through the roof, but user engagement remains lower than we’d like. More experiments are due in 2020.

Civic Engagement Day

• What it is: On Election Day every year, I have been giving the product team the day off to a) go and vote, and b) participate in their community in a way meaningful to them.
• 2018 Result: Massive success. 100% of our team voted, and participated in charitable activities like bake sales, school support for their kids, or volunteering at polling places.
• 2019 Updates: Still a success though tweaks are needed. Mainly, awareness of election day is low, so we need to plan this out well in advance to make sure everyone can plan to do a volunteer activity in their area.

SWAG

• What it is: I was notoriously averse to swag culture in my time at Capital One. However, I’ve turned a new leaf. Little things like stickers and t-shirts can go a long, long way to building a team culture.
• 2018 Result: Positive, but not revolutionary. The swag we got (team logos, team stickers, and enterprise-wide swag) helped the feeling of inclusion.
• 2019 Updates: More stickers, and hoodies. The stickers remain more fun for me than others, but they remain a cheap way of exporting culture to the rest of the org. The hoodies have been a huge hit, and actually were way cheaper than I expected.

2019 Experiments

PM Small Council

• What it is: One of the best things about my team is the accountability we take as PMs for overall product performance and health. With that, comes a spirit of openness about feedback. The Small Council is a sprintly meeting, with lunch provided, where the PMs take turns educating each other on various topics. Recent topics include: how to run better refinements, what constitute good acceptance criteria, what you need to know as a PM of an AWS product, and why PMs experience Impostor Syndrome.
• 2019 Result: Massive success. This meeting has continued to foster the spirit of vulnerability that allows for people to safely prioritize learning. It has become our team’s favorite

Virtual Huddles

• What it is: In 2019, we removed the Daily Huddle on Fridays, and instead conducted it virtually (over Hangouts in our shared Product/Tech channel).
• 2019 Result: Successful, but with drawbacks. Some people prefer this because of the asynchronous nature, but lament the lack of interactivity that we see with the daily huddle. If you can make the daily huddle work (even if only 3x a week) that’s a better approach.

Remote Employee Office Hours

• What it is: With co-located teams, it’s easy to have organic conversations on any topic. With remote associates, the barrier to picking up a phone/starting a video chat becomes much higher relative to walking down the hall. To get around this, we started office hours only available to remote associates. It’s a standing block on calendars, and the remote folks can accept/decline the day of the meeting depending on whether they have stuff to talk about or not.
• 2019 Result: Moderately successful. This has definitely increased communication between me and the PMs on our team. However, it’s a bit stodgy, and not precisely organic like the in-office conversations.

Asynchronous SAFe

• What it is: I’m a notorious opponent of SAFe. I think it’s slow, clunky, and allows for laziness on the product team toward transparency and agility. However, our team experimented with it in early 2019, and iterated on it over the course of the year. We ended up in a place where we do very small quarterly planning (for the purpose of signaling to our partners) and then use our sprint updates to communicate changes to the plan. This seems to have minimized the need for full-scale PI Planning sessions, while maintaining the communication value SAFe offers.
• 2019 Result: Mostly positive. We’ve still got room to improve in terms of our product team communication with our partners, but the expense (time and money) of the PI Planning session has been removed, and the accountability of transparency has been shifted back onto regular, sprintly routines rather than quarterly ones.

Maternity Coverage

• What it is: This will be the subject of another post, but in brief, we experimented with dividing up responsibilities for a PM on my team while she was on leave. During this 12 week period, we split up her responsibilities between me (story writing, research, partner engagement), the tech lead on her team (leading refinement), and the Scrum Master (running ceremonies, board management).
• 2019 Result: Highly educational. By serving as the PM in a company where I entered as a Director, I learned a ton about the experience our Dev Teams and PMs have here. However, the time commitments involved really meant that I was a sub-par PM, and a less engaged manager. This “worst of both worlds” approach probably means that we should divide up coverage differently the next time a PM is out on leave.

Performance Management Frameworks

• What it is: Building a standard set of benchmark behaviors (which leverage the McKinsey model), and a vision for what successful product management looks like. Pair this with a similar set of expectations for our Scrum Masters to try to drive efficiencies within the Dev Teams.
• 2019 Result: Promising, but it’s early. Rolling these frameworks out has given us the vocabulary to talk about performance in an unemotional way. This has also served as a useful tool for regular performance management conversations, as well as long-term goal setting and development plans.

Enterprise Wide PM Channel

• What it is: Continue to export the culture of product management across the enterprise by establishing a Slack-like channel to share interesting articles about PM, and seek advice about challenges we’re having.
• 2019 Result: Unsuccessful. The channel doesn’t allow for high fidelity conversations on sensitive topics, nor engaging presentations on successful projects. We will discontinue investment in this for 2020, and try other approaches below.

2020 Experiments

Instrumentation Sprints

• What it is: Taking a sprint (or half-sprint) dedicated on instrumenting one of the tools we use. This won’t be bullet proof, but a successful outcome will result in us establishing a pattern for continued investments in instrumentation.
• 2020 Goals: Moving toward having feature-specific dashboards to better measure product performance and diagnose production issues.

Rotating PMs/SMs across Dev Teams

• What it is: Rotating our PMs or Scrum Masters across Dev Teams for a short period (e.g. a quarter) to get a sense of how other teams work, and what tweaks might be made to their own teams.
• 2020 Goals: Building a culture of continuous experimentation for how our teams work.

Enterprise Wide PM Engagement

• What it is: Build a regularly occurring forum for PMs to share their experiences, successes and struggles so that other PMs can benefit from their learnings.
• 2020 Goals: Have such a forum at least quarterly, with >15 PMs engaged per session.

What experiments did you try this year? What are you going to try next year? Please share this with folks you know, and comment about your own learnings!