From output to impact: Lessons from a Coca-Cola data scientist

Episode 173: From output to impact: Lessons from a Coca-Cola data scientist

A lot of times people would ask, ‘Hey, what’s a good target?’, ‘How many emails should we be sending?’ There’s no one target. We’ve been down that road. It’s going to be different for everyone. But the more we can start to understand what these different profiles look like and how those correlate – or not – to one’s overall experience is where I see the enterprise and that intelligence layer evolving as we continue to innovate in the technology space.

Felipe Archila
Director, Product Delivery – Data & Analytics
The Coca-Cola Company

Nancy Goebel
Many of us have spent years trying to answer a deceptively simple question. How do you know whether your digital workplace is actually making a difference? Our guest today has been wrestling with that challenge at one of the world’s most recognized brands. Over the past few years, Felipe Archila has helped The Coca-Cola Company build a sophisticated analytics capability designed not simply to measure technology usage but to understand digital behaviours, capability development and even business impact. Today, Felipe is bringing those same skills into a new leadership role in data and analytics inside of Coke’s finance division. In the conversation, we explore what every digital workplace leader can learn from that journey.

This is Nancy Goebel, your host and DWG’s Chief Executive. Digital Workplace Impact is brought to you by Digital Workplace Group. Join me now in conversation with Felipe. Happy listening!

Felipe, I am just so excited to welcome you to the studio for a chat. This is a podcast that’s been a long time in the making. It is the best of a celebration as well as a moment of reconnection. And so let me start by saying thank you for joining me in the studio – and welcome.

Felipe Archila
Thank you for having me, Nancy. I’m so excited to have this conversation. Like you said, it’s been a long time coming and in my just over five years of knowing you, I feel like I’ve known you for much longer, which is really saying something. So I’m so excited to dive in today.

Nancy
And so, over the years within DWG’s circles, you’ve been very generous with sharing your journey as someone who is expert in challenging all of us as digital workplace professionals to think about decision intelligence and outcomes by bringing together the right measurement strategies. And so, knowing that you were making a career pivot this spring, I wanted to make sure that we had a chance to draw out some of your insights from your time within the digital workplace team at Coke and then to celebrate your new role – because you’re about a 100 days in now.

Felipe
That’s right.

Nancy
And see what bridging points that gives us between what you’re doing now and those digital workplace routes. Does that work as a plan of attack with you?

Felipe
Absolutely. Yes, it was a little over five years ago, and we were still very much at the height of reimagining work in this new COVID era that we had just entered the year before. And I started my career as a practitioner – I like to get my hands dirty as a data engineer by trade. I’d been well-versed in all the technical aspects of being able to gather large amounts of data and to present it however we wanted to. But this role in the digital workplace was the first time where I needed to make strategic decisions from that data and to be able to put it in a format that spoke the language of the business. And it’s not obvious, because from the beginning, it’s tempting to just go into your admin console and to run a report on the number of emails you’ve sent or the number of Teams chat messages that the company has sent as an organization. But what those signals really lacked was the context and the real work behind those activities.

And so, as we thought about a measurement strategy, we wanted to at least start with what we had and to understand: What does this get us now? What is it saying about the behaviours of people who are primarily used to working in person who have now been thrown into a remote role, not by choice, and how they’re trying to keep up given the tools that have been made available? And it’s possible that they weren’t aware of all the features to help make life easier. So that was the whole raison d’être, right? The whole reason the digital workplace team at Coca-Cola was formed was to really get a handle around the business context and the enablement of those tools. So we had to start with what was measurable now and then continue to identify those opportunities. Well, does a salesperson interact with the tools a little bit differently than someone who is in finance or in marketing or in the technical space where their role is completely different? They’re creating all the flavours that that we enjoy.

And so, getting very specific about the personas and understanding what their relationship is with the tools and how are we best serving them comes through partially by seeing patterns in the data, but we also quickly realized that there was an element of experience that the metrics alone weren’t able to capture. So being able to implement feedback loops and to really understand what the key drivers are in someone’s digital experience based on whether they’re having a hard time finding something on the intranet, or if they have no idea how their coworkers like to interact online if it’s an email or a chat or a channel message.

Or, you know, now, as of the last three years or so, right, how many people are actually leveraging everyday AI to help make their lives easier and which ones are taking the next step to productionize or to create an agent to really think about their organizational structure and their goals using the tools.

So, I say all that to say that measurement has not been one static thing. It’s evolved as the technology has evolved, but what we’ve always tried to keep at the centre is the people and the jobs to be done. and a lot of times people would ask, ‘Hey, what’s a good target?’, ‘How many emails should we be sending?’ There’s no one target. We’ve been down that road. It’s going to be different for everyone. But the more we can start to understand what these different profiles look like and how those correlate – or not – to one’s overall experience is where I see the enterprise and that intelligence layer evolving as we continue to innovate in the technology space.

Nancy
Yes, and I think it’s one of the hardest things for – whether it’s DWG in a benchmarking capacity or you in your data engineering role – to go back to someone and, you know, it depends on when they’re looking for that magic number. And when you watch digital workplace programmes mature over time, you often see that there’s a paradigm shift from focus on measuring usage to then seeing that process of discovery that you described from taking advantage of what’s at your fingertips like usage stats to then really starting to use both quantitative and qualitative information to guide decisions – both strategic and operational – in evolving the value and the impact of the digital workplace. And so I’m excited to have a moment to put my nerdy propeller hat on with you for a little bit in a conversation like this.

And, of course we’re catching you at a moment in time where you’ve made a change in focus. In looking back, what would you think you’d land on as the most important steps you’ve observed in moving from reporting metrics to creating insights that business leaders can actually act on?

Felipe
Yes, of course. I think the biggest thing for me early on in my career was I really resonated with the philosophy that data can be treated as a product. So, putting that product mentality onto it, so that there are specific owners, there are domains that you consider in the data, that you have a roadmap to understand which increments of value you can deliver today, tomorrow, beyond, and not treat it as say like a delivery of a report that supposedly runs weekly and then you check in months later and you realize that the refresh had broken, but nobody noticed because no one was actually reading the report! It needs to meet people where they are, and that takes continuous feedback in that flywheel. That’s typical of any product organization. For me, that mentality, that mindset, that’s guided me through over the last few years is exactly why this opportunity I have now within the finance organization really works because I’ve been spending the last 100 days really trying to understand the finance function.

I don’t have a finance background – I have more of a product owner background, which then allows me to have very targeted conversations about going in with curiosity over what a typical process is for, say, your weekly, monthly, quarterly routines. What data is it that you need at what time? What decisions are those enabling? How are you getting that today? And trying to leave it open-ended.

And even if there’s a whiteboard, I’ll try to sketch out my rough understanding of what it is and then add little blanks to say, How much time is that taking? How many cycles? Where does some brain power come in that is not reflected in any kind of system that we can start to capture and identify where we can make life just that little bit easier?

Nancy
Uh-huh.

Felipe
And that exercise goes a long way in establishing trust and knowing that you’re translating the business context and the business language to the technical requirements for be it a data lake or a chatbot or a power BI reporter, or whatever the end product happens to be, I try not to go into it with a specific solution in mind because it is going to depend. And that’s the kind of conversations that I really enjoy having because I’m a lifelong learner; that’s my number one strength according to some, you know, personality test I took! And it’s a certain point where this was a function I knew the least about and I was really eager and excited to jump in and get my hands on it.

Nancy
Yes. Well, I love that a conversation like this reminds me that we are kindred spirits. and I especially think that natural curiosity is not only what allows you to shape-shift from your previous role in the digital workplace programme to now finance, but also, broadly speaking, when I stop and think about the challenge that generative AI has brought to the workplace, it is asking – dare I say challenging – all of us to become even more naturally curious because the strength of the answers comes from the strength of the questions that we ask.

Felipe
Exactly.

Nancy
But also, to put aside human nature that sometimes resists change and to be open to possibilities as we start to bring together human and artificial intelligence in a co-working environment to create those leapfrog moments in the organization – so whether it’s better process, better decisions, etc.

Getting to the value story is kind of the challenge of the moment – and we talked a little bit about this during the DWG Member Meeting in the spring when we came together, and we put it under the banner of ‘Going from vanity to value’ and looking at the evolution of measurement. So, I’m curious to see what you’ve learned so far in your journey about measuring outcomes over activity and any wisdom you can share there.

Felipe
Absolutely. The biggest thing that comes to mind for me is what I was starting to do towards the end of my time in the digital workplace was exactly that – it was not to take the usage metrics at face value, but to understand what sorts of behaviours or what sorts of outcomes those could be indicative of – in combination with other things.

And I’ve heard other member companies talk about this in relation to their intranet, where they’ve moved beyond page views, right? And they’re now focused more on things like dwell time or engagement or likes or comments – that kind of nature to understand that people not only landed on the page, but did they stay on there long enough to have consumed the content? Say, if there’s a 30-second read and people spend an average of 10 seconds – that’s just a very simple example – but trying to come up with ways we can quantify an outcome in a way that can be used as a proxy while we start to iterate and figure out what the true outcome or the true pain point is.

And I’ll give another example of earlier, which is actually a learning I would say for those practitioners out there. As we were starting to come up with a Teams channel strategy – so understanding what to use, when, and how, and we had started a Teams bootcamp initiative that we wanted to help people understand, ’Okay, here’s how you use the channel structure and here’s when to use a chat versus a channel message and here are the benefits of those’, my early idea was to come up with a high-level teams engagement score, so some kind of index that took a formula that basically weighted channel messages and replies more heavily than chat messages or reactions. And there was a proprietary – a secret – formula, if you will, because we love those!

Nancy
Yes.

Felipe
And it came up with a score. Now, the learning from that was it was very easy to game the system. So, it’s like, well, channel messages are weighted a lot, so we’re all going to send 10 channel messages a day with no regard to the quality, but we’re just running the numbers up. And so then I went back and revised the formula and I said, ‘Okay, well now there’s going be an X factor or there’s another weight to it based on the number or the percentage of your channel members who are active.’ So that helped level the playing field a little bit.

But I fast forward one or two years later and I was having a conversation with a senior leader within the technical space and she said, ‘Oh yes, we use Teams all the time. And frankly our email or our meeting counts actually went up!’ And one of our early hypotheses was that meetings will go down, but she said in reality the meeting counts went up – but the quality of those meetings was so much higher because we all knew exactly what knowledge base we were working off. We all had the same context. We all knew what the next steps were going to be. And those meetings were less about catching up and more about decision-making and insights and strategy.

And if you look at their engagement score, their index that I created, they’re not like top at the top, but it was the quality of those conversations that resulted from having a shared understanding and ways of working that worked for that team – and that was a real ‘aha’ moment for me. That’s one of those things that’s going to be constantly evolving. And as you communicate with your stakeholders, it’s one of those that you really cannot discount the experience part of it and going off of telemetry alone. It’s not going to tell the whole story.

Nancy
Yes, I love this reflective moment because it puts the concept of data and analytics in a learning and discovery mode that can and should be evergreen. And so, to do that within your own team is important – it’s necessary, it’s healthy – and bringing your stakeholders along that paradigm equally so, if not even more, important. And sometimes when executives are thinking about workforce analytics, they can have misconceptions about what that process is supposed to reveal. Do you have any thoughts about what some of those biggest misconceptions can be and what they should be focused on instead?

Felipe
Yes, I have many thoughts!

So, one of the things that we had to really think about balancing early on was the idea that digital workplace analytics, the purpose of that is to ensure that our people are equipped to get their best work done using the tools and platforms made available to them – and nothing more. And it was a process to get people to buy into it. It was one of those disciplines that I wouldn’t say necessarily existed at Coca-Cola before I joined. It was primarily run usage reports and, you know, ‘Here’s the chart and we’re sending more emails this month.’ Okay. What does that mean? There’s no sense of what is good and what exactly is going on with that.

But the more we started to uncover different dimensions, different slices of the data to say here’s how different functions behave differently, the conversation could very easily have gone into, ‘Well, is this surveillance?’ ‘Are we looking at the content of what’s being produced?’ ‘Are managers trying to get their hands on this to say, “Well, there’s no reason this person could be burnt-out because this is their activities”.’ That is not at all what we were trying to accomplish.

And, one of the things that was really key is establishing that partnership with our legal and privacy teams to co-create what is going to be the policy and the vision of how we leverage these sorts of analytics. Because, on the one hand, I’m willing to bet there are people who wish we knew a little more about their pain points, and, to that extent, I was having a conversation with someone who’s leading the learning and development space. And he’d been here for maybe 90 days at that point when we were having a chat, and I asked him a pointed question. I said, ‘Do you feel like we survey our people too much, in terms of getting an understanding of the voice of the customer kind of thing?’ And he said, ‘Yes, but we don’t ask them the right questions enough.’ So, there’s survey after survey coming out on like ‘Did we close your ticket and were you satisfied with that?’ But those kinds of things come at you so often that where I think we have a massive opportunity is in being very precise about where we focus our questions and how we can then tie that into something broader, like ‘It’s hard to get things done’ or ‘I can never find what I’m looking for’ or ‘It’s a nightmare to get approvals.’ You know, not necessarily things I’ve heard here, but these are the kinds of things you probably hear throughout all the member organizations as common pain points.

Nancy
Absolutely.

Felipe
And so it always has to be with privacy first and foremost, understanding what is the minimum amount of information we would need to be able to address holistic challenges that we’re seeing while still being precise enough to know that my job looks very different than my colleague’s job just two desks down. We have very different profiles. There’s got to be some level of communication and trust that what we are doing here is truly with the person in mind first and how we can enable them best.

Nancy
I love that vignette. It is so powerful and chock full of great learnings. And I’m going to pull the thread a little bit more and bring AI into the mix, because as generative AI becomes embedded in everyday work – not just for the digital workplace teams but the enterprise at large – how do you think the digital workplace measurement agenda needs to evolve along with that?

Felipe
It’s a timely question because we just had a summit where we talked about it this week within my new organization. I think we have to first understand what people’s current understanding of AI is. So, there’s obviously the first step, which is knowing that we’re talking about generative AI versus the more – and I’m putting air quotes around it, the ‘classical AI’ – which involves machine learning and regression models we’ve had forever. But some of the novel ideas, and even with agentic now, is what is people’s base understanding of AI? And, to some degree, it’s encouraged – at least it was encouraged this week, and I’ve been thinking about it for myself – is how do you know what you don’t know? Is there some way you could tell if there is a certain kind of task that you spent the most time on, or really understanding end to end what the work output is – and trying to get to a place where we can iterate little by little to say, ‘Frankly, I don’t really enjoy creating PowerPoint decks.’ Okay, I’m admitting this on this podcast. Everybody now knows.

Nancy
(Laughs)

Felipe
If you see a PowerPoint deck from me, it was not made gleefully, but over time I’ve started to notice that the capabilities of creating a PowerPoint deck has gotten better. So, you have to know thyself and, know there may be some things that don’t apply to everybody. Like I was talking to somebody who has a very strategic task that she has to do that on face value seems repetitive but actually takes a great deal of nuance and know-how and domain expertise to be able to do. And so, the gut reaction is well, just automate that, but it’s not quite that simple. What this person could really benefit from could be the automation of the research and analysis on changing tax laws or evolving privacy laws or things that kind of feed into that know-how so that she’s not having to go out and do that research and then apply it.

Nancy
Yes.

Felipe
And so what areas truly still require that human touch? I’m very much big on the human-led sort of rethinking of work to where we can free up more time to consult, to be more human in leadership within our teams. And again with every one of these I have to put in the in big red neon lights that AI is a tool. It is not a strategy and it’s one of those things that it’s easy to say, well, let’s just put AI on that. But at the same time, again, going back to my conversation about really unpacking and understanding a process end to end, and being curious and trying to see what makes sense to still have the human judgment and what really could be automatable and really build your core that way rather than trying to automate for automation’s sake. At least that’s how I think about it!

Nancy
Yes, and it’s been interesting if we step back and think about the membership cohort and even our wider industry circles, we’ve seen a shift in thinking about this as human in the loop, gen one, to human in the lead, gen two, because there are those things that AI, yes, can support and enable, but there are also soft skills that will become even more important as AI and human specific co-working starts to mature. And so just thinking about the fact that you have made a role shift, it’s a nice bridging moment to say, based on your experience within the digital workplace programme at Coca-Cola and your recent move into a broader data analytics leadership role, what skills have proven most transferable for you so far?

(Pause)

Oh, this one made you think! (Laughs.)

Felipe
It’s so funny because all the skills I can think of are the soft skills. So, listening, the translating of what I’m hearing and what I know to be true and can think two, three, four steps ahead as far as what you’re saying now has, you know, downstream impacts, that sort of idea.

I think, at the end of the day, we are all humans here working for this company and we have these titles to our names, but even just taking a second to ask about something that’s happened, maybe a follow-up on a quick question someone asked, or even something like if my kid was sick, if you ask how my kid was when I’m back in the office, like that automatically gets 100 points in my book. But the idea is that we never stop learning. And that goes from going from the inside and to the outside. What I love about Digital Workplace Group is that we constantly are bringing the outside in and sharing learnings with other member companies and organizations. I’m a member of at least a dozen other organizations just within the Metro Atlanta area and internationally because what gives me the most value from those is hearing from other leaders and other practitioners who are facing the same challenges and being able to see the through line across whether it is truly a change management problem, a leadership messaging problem, whatever it happens to be, a training problem. But rarely is the problem the technology. It’s about understanding and bringing people along for the journey – how you tell that story about, just generally speaking, why this product matters and how it addresses real problems that you may be facing.

Those are things I would say have helped me the most in making this jump. And again, saying again, I’m zero finance background. I’m coming in here completely with fresh eyes. And I think that’s partially the strategy – they wanted someone who could take that outside view and bring in those other perspectives because there’s some semblance of ‘Well, this is something that we’ve always done. There’s always a routine that happens.’

Nancy
Hmmm.

Felipe
And I’m here to ask why it is that way and could there be a better way, while still respecting that they’re the domain experts and I’m bringing in suggestion solutions –but with their expertise and their lived reality at the centre and not applying solutions blindly.

Nancy
Yes. So, we’re at the point in our conversation together where there’s an opportunity for you to help us end things with either a final reflection or something to inspire those who are listening to this conversation. And so my final ask of you is if you could leave digital workplace leaders with one piece of advice on creating impact and not just measuring it, what would that be?

Felipe
I think, for me, the biggest piece of advice I could give is to continue iterating – start with what you have and start those conversations early, not just within technical leadership, but also within your people, your chief people office – and try to get to a shared understanding of what it truly means to work at your organization. What makes up the experience of working there, both physically and digitally? What sorts of things do you aspire to? Is there anything that you see automatically as an opportunity? And what signals do we think we need to quantify that? It doesn’t have to be complicated at first. You know, I had the engineering background, so I could easily connect to APIs and get all the things I wanted. But in the absence of that, co-creating the strategy from the onset with your measurement, your intelligence, your analytics lead, whatever you want to call it, and establishing what those KPIs are early on is going to go a long way than bringing them in at the end and say, ‘Alright, we’ve done the thing. What do we measure?’ That co-creation is critical. And it has to go across disciplines too, because the digital workplace is the employee experience in my eyes. And I think there is value inherent to that, that we that we do bring value to the business and in time saved and feeling like you’ve got all the knowledge you have and, just as I spoke about with my friend and colleague in in the technical space, that they had much more productive quality time together because they had laid a foundation that the digital workplace had enabled for them.

Nancy
Yes. And so, you know, we started this conversation around ‘Is there a magical solution to what you should measure?’ And we’re closing on the idea of outcome. And I have often said to people who come to me with the same question you heard time and time again, ‘What is the perfect thing we should be measuring?’ And I said ‘It depends.’ I immediately follow it up with ‘Begin with the end in mind. Understand where you’re trying to get to, and then the measurement needs to flow from that.’ Don’t get enamoured in usage and stickiness. And, you know, there are things you need to measure at the beginning of a process, just to make sure that people are showing up for what you’re trying to create.

Felipe
Of course

Nancy
But that’s not the end goal; that’s just a stepping stone metric. And so this conversation really ties that up with a lovely bow.

So, thank you so much for stepping out of a busy day to be in conversation with me, with us. And, of course, we look forward to following your career travels as you find your sea legs in this new role and who knows where beyond? But thank you so much for your time, Felipe. It was just wonderful reconnecting with you through this session.

Felipe
The pleasure was all mine, Nancy, and I really look forward to the next time we get to chat.

Nancy
Same here. So we’ll put a pin in it for now – to be continued at the earliest opportunity. But thanks again, Felipe.