Building a Scalable Content Engine at IBM by Eda Kavlakoglu of IBM

Eda Kavlakoglu, a Program Director at IBM, joined us for a webinar on how IBM builds a scalable content engine.

She shared their four operating models that guide their teams:

• Workflows

• Staffing models

• Keyword research

• Writing and content creation

She also shared how her team measures success including feedback loops for both business and individual performance.

Listen to the webinar as audio only on your favorite podcast platform:

• Apple Podcast

• Spotify

About Eda Kavlakoglu:

As a Program Director within the ibm.com content discipline, Eda Kavlakoglu leads the development of inbound content at IBM. Using keyword research, she and her team prioritize, write, and edit content that ranks on page 1 of Google Search, driving organic traffic growth for the business over the last three years.

Follow Eda on LinkedIn: https://www.linkedin.com/in/edakavlakoglu/

Eda: My name's Eda. I'm a program director for the ibm.com organization. And I'm just going to tell you a little bit about our story about just creating an inbound content engine at IBM. We're, you know, still in the process; we're still growing and learning every single day. But I think we have some things that we can share with, you know, The broader, I don't know, world about how we've kind of set up our team and processes.

So maybe this will be helpful to you, and if you have questions, I'll try to leave some time in the end to answer them. So before I kind of tell our story, I just wanted to kind of acknowledge the fact that I b m is kind of uniquely positioned to win. In SS e o and there's kind of two reasons for that.

One, the i B M as a brand has some credibility. We are one of America's oldest and most recognized brands, and we have a longstanding history of innovation. You know, despite what Wall Street may say of like the last 10 years and. Being an old company also has its benefits and perks. We also have an old website, and because we also are a fairly large company, we can acquire new companies which come with their own websites.

And so that can assist in a backlinking strategy that can help us boost Boost our kind of ss e o rankings overall. And so I think it's just important to kind of note that before I kind of take you through the processes that we use at I B M.

So to kind of ground everyone, I think it's important to just talk about kind of the business problem, the goal, the approach, and, you know, the KPIs that we're using to measure success. So like many companies, I b m kind of disproportionately invested and paid In paid media campaigns to drive traffic to ibm.com.

This isn't kind of unique to I B m. A lot of companies do this. And so my boss, Brian Casey, kind of noticed this and decided that he wanted to increase organic traffic to ibm.com and kind of reduce IBM's dependency on paid media campaigns. And we weren't kind of very necessarily original in our thinking.

If you look at kind of ss e o like blogs and other content that's on the web, a lot of people will talk about these topic clusters. And HubSpot has a really great article on topic clusters. And so what we were looking to do is kind of apply The topic cluster examples that they had through a B two B lens.

So maybe just kind of tweaking it slightly. So instead of talking about kind of workouts and then maybe all the different workouts that you might do, like arm workouts or ab workouts, et cetera, you might take a topic like machine learning and kind of identify what are all the subtopics that kind of fall within that umbrella or that kind of That kind of keyword area.

And so we kind of do this process of kind of keyword research, keyword mapping, content research, content validation, editorial and content staging and publishing across our organization. And then, to evaluate the effectiveness of our program, we look at organic traffic and pH search rankings to see, you know, are we kind of writing authoritative content?

Are we winning in ss e o, and are we kind of overall driving? More traffic to ibm.com. So I always think that it's easiest to understand a process when you go through an example. And so, for this webinar, I just wanted to take the example of machine learning and try and break that down for this audience.

So we. We use keyword research in order to understand the ecosystem that exists around a major topic. And so, a topic like machine learning generates over 2 million global searches per month. And so it's a huge category. And as you can imagine, within that category, there are a lot of kind of long-tail keywords, kind of sub-intents that exist underneath.

And some of those include, this isn't like all-encompassing, but. You can see what is machine learning, machine learning definition. So that's kind of informational kind of content around the definition of machine learning. Then you have kind of comparison-type content. So AI versus machine learning and vice versa.

So people are trying to understand the nuances and differences between those two categories. You have machine learning, python, so people are actually trying to get a sense for. Sorry, I'm just gonna. Actually, I just realized the camera on my other screen was still on. So so, then you have machine learning Python, and you have people who are actually trying to get hands-on with it.

Within machine learning and actually trying to write code within Python to execute machine learning models, then you have kind of large language models. And you know, that's like the latest and greatest within machine learning these days. And then you also have kind of machine learning courses and machine learning certifications.

And so you can see with each one of these keywords. They kind of map to a different intent type. And those intent types can actually map to a different page template that you might have within within your C m s or within your design team. Essentially like templates, page templates are how you kind of scale content across your organization.

And so. So, you know, I kind of went through this example of machine learning, but you can kind of see how each one of these different keyword types will kind of map to a different template. And so for, you know, what is machine learning and kind of definitional content will create something that, that we call a learning page around that.

For kind of comparison type content, that might be a sub-content of a learn page. But if you wanted to go deeper, we might create a blog that's kind of really focused on that. That topic and the nuances between them. For a kind of machine learning, python, you could imagine kind of a code patterns type page template that might exist to attract a more developer-oriented or data science audience.

And then, finally, if you had like machine learning courses and machine learning certification, you could imagine that kind of be also being bundled together in some kind of training page template where You're actually creating course materials for people that you want to arrive at your website.

I'm gonna for the purposes of this example, I'm just gonna focus primarily on learn pages because that is kind of what we've been explicitly like, or what I've been focused on primarily over the last couple of years. So the next step, like once you've determined that your kind of intent type is kind of definitional in nature, And you're looking to build an alarm page.

The next step is kind of content research, and there's kind of three things that you wanna think about when you're kind of conducting your content research. It should be very kind of labor intensive. Like you want to kind of really understand the competitive space space within this topic.

So looking at. Every page that's ranking on page one, like what are the content patterns that exist there? Understanding, like your market research, what, how is the market talk about this topic? Thought leaders, there's academic research that we also read largely because some of the. Other content that exists on the internet today is just not keeping up with the pace of innovation within that space.

You have book expert excerpts and even like, you know, we watch videos on YouTube because that has basically become like the new form of like like the New York Public Library. And so I think it's just like important to be able to summarize and distill all the important aspects of the topic for your au audience.

Once you've kind of done that, then the next kind of step is to really provide a company-like point of view on the topics. Particularly if you're a company like I B M. Where when we assign a topic, like chances are we're actually not the first person to cover that topic within I B M. And so you know, we should be looking at what have we written before you know, what, what are existing kind of pages that exist within the ibm.com domain.

We should look for SMEs to kind of vet our ideas. Thinking with, and then also our product teams to understand, okay, how, how are we actually trying to go to market within this space? And then finally, once you've done that, You'll want to think about kind of adding something novel to your content.

Give people a reason to come back to your website if you're just kind of regurgitating everything that exists on the internet today. Like, why should I come to your website? And so you know, it can be as simple as just providing a clearer definition of a topic. Sometimes a good example is chatbots.

A lot of people confuse The difference between like, chatbots, voice assistants, and virtual agents. So if you can provide a really clear definition for kind of that topic, I think that's something that's valuable to your end audience because now they'll understand those nuances between those two those terms.

You know, another way that you can add novelties, you know, new facts or data that maybe wasn't. Available before. So if you have some kind of user research team that you can partner with, I've seen kind of websites, and I, I think we've done this across I b m as well, like kind of launching our own kind of research that we can then actually publish some of the facts or like data points that we've gained out of that research.

So just kind of finding a reason for your reader to come back to your website is also important. From there, then we validate that research, and that's where clear scope comes in. It's really been an invaluable tool to our content development process for a couple of reasons.

One, it. It kind of allows writers in a more self-directed way to understand whether or not their content research has been comprehensive. And so you can kind of start to see, all right, did I actually encompass everything that I was supposed to do within, like, the topic of machine learning? And, like within our page day, we might need to update it to actually reflect some of the latest and greatest That's been going on within this space around generative ai and large language models.

And so clear scope can actually help identify, okay, there's actually like a, a new section that we need to create on this, on this page to make sure that we're encompassing, like the full breadth of this topic. It also provides direction around page structure. And so, like, you can kind of see this outline feature over here, so it can validate whether or not you're kind of structuring your page in an expected way.

It also just identifies data-driven, actionable ways to improve your content. Like, I think when we think about content teams of the past, a lot of it is maybe like you think about a newsroom and people kind of going around and like shooting ideas around how to make something better. And what I love about Clear Scope is that it really provides an actionable way to make your content better so that it's going to rank.

Within Google search. And then four is that it provides a bar for writers to achieve. I love the feature of a content grade, and so it allows me to kind of manage my time more effectively, too, as a manager. Where I don't want to see in a writer's article until they've actually refined it enough so that it's at a plus grade.

And of course, like there are ways to kind of game any tool so that, you know, maybe you get a plus grade where the quality of the content isn't at where it should be. But it allows me to at least kind of set that threshold with my team so that they're not kind of sending me something that Is not kind of ready to be reviewed.

So from there, once we've kind of validated our research using clear scope, then we kind of move into editorial. And what I've kind of typically seen is that editorial typically falls into four main categories. One is kind of fact-checking, And this is gonna become kind of, I think, more important as we see the artificial intelligence space.

Like, expand and become features within existing products. And as new products kind of spin out as part of this new technology like, but even before kind of generative AI kind of became this thing, we still needed to kind of go through this vetting process of making sure that what writers are kind of putting together is actually factual in nature.

And so kind of vetting content for misinformation is, you know, Now an even more important part of the process than it was like six months ago. Then kind of another type of edit that we frequently see is kind of logical flow. So sometimes, maybe in a writer's head, they're kind of, they think that they're kind of connecting to separate thoughts.

Well, sometimes they just need an additional sentence to kind of elaborate on that point more clearly. Or sometimes they just need a transition to kind of help improve the flow of that that article structure. The third kind of category of edits that we see is grammar and spelling. So you know, yes, we have tools like spell check, but, you know, everyone, I think, sometimes makes a, a small error, and that's not necessarily how you wanna show up to the rest of the world, particularly when you are a large brand like I B M and then also sometimes English is not the first language of some of your writers.

And so when you are managing a global team, it's also important to help Help them kind of structure their sentences in ways that are logical to the broadest group that you're trying to attract to your website. And then, finally, you know, we talked about how templates are a way to scale.

Page templates are a way to scale content across an organization, but also, you know, Different organizations have different kinds of style guides. And that is a way to kind of also scale the type of writing that you want. So, i b m has different rules in which they kind of expect writers to write.

So a good example is, you know, maybe you write with the Oxford comma like I do. But IBM's Marketing department does not actually use Oxford commas. And so that is kind of a rule within our style guide. And so if you want to kind of scale that kind of like practice across your organization, you wanna make sure that it's documented for people to kind of review and kind of know as they're developing content.

And then kind of the last step is kind of staging backlinking, QA, and publishing. And so after your content has kind of been reviewed and gone through some kind of editorial process, the writer's going to kind of stage that con content and then also start to kind of backlink or crosslink content across your website and backlinks are I kind of mentioned earlier, backlinks are a critical component to search.

You know, you can write great content, but if you don't have backlinks to kind of support it. You'll, you'll never win in search. Or at least that I, I don't believe that you will. And so, there's kind of two ways to generate backlinks. One is just internal backlinks. So you can just make sure that you are improving the findability of content throughout your website.

So if you have related content within your article, you'll wanna make sure that you're embedding text links to maybe other learn articles that exist in your website. If you're promoting products that are relevant to the topic that you're talking about, you wanna make sure that you are also driving to those products so that people can find your products.

And if you have like broader solution pages like maybe the topic itself is, is very broad in nature, you'll want to kind of drive to those solution pages or, you know, if you have like a training course that's available, you know, you get the point. So there's kind of backlinks that are kind of somewhat within your control from an internal backlinking standpoint.

But from an external backlinking standpoint, that's kind of less within your control where you really just, it's kind of a waiting game. You just need to kinda wait until backlinks accrue around your content for it to kind of move up within search. And so when you're kind of starting off and determining what topics you wanna go after, I think it's important to kind of use keyword research tools to understand what.

What's the backlink profile of the top kind of ranking content across Google search? Because, like for the topic of machine learning here, you can actually see that the number of backlinks that exist for everything that's on page one is actually incredibly high. It's a lot higher than other topics, and so you can expect that it's actually gonna take a long time for You and even if you are a big brand, to rank for this topic because you're competing with a lot of pages that already have a lot of external links or internal and external links driving to it.

So I think when you're thinking about setting expectations with maybe other parts of your marketing organization, this is also important to kind of Look at, at the onset, so you can level set with people around, you know, how much time do you think it might take to kind of rank for this type of term.

Okay. So I'm gonna kind of switch, I kind of told you about our process, and I want to switch gears and talk about kinda operational workflows of how you can, can, can execute against this. And so, There are kind of two models that we've used at I B M, and I definitely have a preference for one over the other.

But the first kind of workflow or first model that we've worked with is like agency models. And I think this is pretty standard across a lot of large companies, is you'll kind of work with an agency to kind of outsource the content. But. It has a lot of overhead when you do that. So as you kind of look at this sample workflow that I've kind of created here, you'll kind of see that, like, you'll still start with like, you know, keyword research, kind of creating a, a content brief for the agency.

But within the agency, there are kind of these, their own layers that exist. They'll have their own kind of writer, their own editorial team, and a project manager that will help kind of guide that content process. And that just adds. More time to get a single article out the door. And so you'll kind of go through that kind of process of like writer, edit, like agency writer, agency editor agency, project manager first before it even gets to you on the client side.

And then you have to kind of replicate that same process on the client side as well. And so you can imagine how much longer that overhead will take to ship an article. So. I, I'm, I'm not a huge fan of agency models, largely because it's not a great bank, bank for your buck in the long term.

So the other model that we kind of work with is in-house an in-house writer model. And the reason why I love this is because it's just a more streamlined approach. And before, well, actually like to take a step back. You can bring writers in-house and still work through, like, with agencies, freelancers, and contractors, but.

You just wanna kind of be sitting side by side with these folks because it's going to be more streamlined in terms of the feedback that you're giving them. And so you'll be able to build domain expertise within your team over time. Actually, not even just domain expertise, but also writing expertise.

So like the, the ways that you know, your company rights, like, that's also a learning curve as well. So it's, it's just like, and I, for me, like. Or not even just for me, but for kind of search in general, like time is a valuable resource because the sooner you can get get the content published, the sooner you can actually see it.

The business value of that content. And so, like, time should actually be looked at as a critical resource within the game of search. And so you're, you're able to kind of win and search more often and, and more quickly. And this is also important for when you're using clear scope as well, because when you think about the reports that you're generating within that tool, the Clear scope report will actually change over time because the kind of search rankings aren't static.

And so you can imagine that, like, your clear scope report is gonna actually look somewhat different a month later than it does a month before. And that's to be expected. And so to kind of maintain your kind of grade and kind of be competitive in this space, you wanna actually be kind of working in, in a nimble fashion.

And then I think we're gonna come up on my last section, which is really talking about success and feedback or measuring success and feedback. And I think the only way that you can do that is, you know, using data. And there's a few different data sets that we use at I B M to kind of understand high performers and low performers across our team.

And. One of those data sets is a project management tool. We use Jira, but you know, there are a number of project management tools, and I imagine they all have similar data points that exist within their tools. But Jira or project management tools really tell you about velocity. And that's just like one kind of.

Dimension that you wanna look at. So, in general, I think it's safe to say that your higher performers do tend to generate a higher volume of content. But I wanna caveat that with, like, you know, generating more content does not necessarily mean that you're generating great content.

And so. You know, one, it can just kind of give you an indicator of who's kind of working quickly. But it can also be this way of understanding changes in behavior across your team to kind of prompt questions for context. So if you see that someone is kind of taking longer than they usually take to kind of Build out a specific article, you can ask them about it.

And but not in a, like, you know, critical way. It's just like taking a more curious approach, like, you know, is this topic more difficult than other topics that you've written about in the past? Which does happen? If you take an example like supervised learning within kind of the AI space and then also maybe like intelligent automation.

Those two topics are very different in scope. Supervised learning is a lot broader. And intelligent automation is narrower in scope. And so, my expectation is that it actually would take longer to actually write an article around supervised learning versus intelligent automation. But you know, maybe someone who's kind of new to the editorial team, they might not know that right away.

And so kind of having this data. It kind of helps prompt these questions. Another kind of Another thing that might come up is, you know if something is taking longer to kind of build-out, maybe that person is also experiencing personal issues. I mean, life happens. So just kind of again, taking a more curious approach or kind of finally bottlenecks process bottlenecks kind of happen all the time, is maybe another team, maybe not partnering as well as they can be.

And so that can allow you to. Kind of intervene and maybe improve the relationship with that team to expedite content flow a little bit more. So once you have an understanding of velocity across your team, the other thing that you'll wanna look at is kind of business performance. And there's kind of a few different metrics that you'll wanna look at.

I'll use our keyword research tools and our web analytics tools to do this. And I'll focus primarily on web traffic for team performance. So we set goals as a team. Increasing overall organic traffic year over year. But I don't wanna evaluate an individual on web traffic performance because top certain topics lend themselves to higher web traffic.

So and that's just because certain topics have higher search volume associated with them. And so if someone got on page one because they wrote about supervised learning and or if someone got. A lot, like a lot more traffic, because They wrote an article on supervised learning versus intelligent automation.

It could just be the fact that intelligent auto or intelligent automation just drives less search volume overall. So it's important to kind of look at the data through that lens of, you know, certain topics or just inherently larger in terms of what they're going to drive from a business standpoint.

But when you're evaluating individual performance, you can look at kind of page rankings. You know, your writer is kind of getting on page one or improving the page ranking of a given page. Where like, maybe you're moving from, I know, page three of Google search to page two. That's still an improvement.

Ultimately, like I, I want everyone to be able to get onto page one of Google search. Then the last. The metric that you'll wanna look at is also content grades. So if you're seeing that content grades are kind of decaying more quickly over time, that'll also tell you a little bit about the quality of writing.

So maybe content that decays faster. It's just not as high of quality as the other content that you have on your website. So, yeah, I, I think that's where I'm going to kind of end our presentation. I kind of just wanted to take everyone through kind of our approach, kind of the different workflows that we've kind of tried out across I B M, and then just how we evaluate ourselves on performance.

And so I'll pause there, see if there are any questions. Travis, I know that you've been kind of monitoring the q and A. Yeah.

Travis: Awesome job, beta. Yeah. We do have a couple of questions; actually, people kind of sent over. So to kick it off, you mentioned internal linking; I think you actually talk about it as cross-linking.

Mm-hmm. Do you use any tools at I B M to assist with that, or is it all kind of manual?

Eda: So I, I do know that if you do a Google search and scope it to like cite colon like your domain, there might actually be a, a tool that I, I'm just not aware of. Like, I actually partner with our s e o team on this.

So I'm not the one that's like specifically doing the, like, searching for other links. But if I were to do it myself, that is probably the approach that I would take is, you know, take a given topic and then kind of do a, a site colon ibm.com to just see what are, what's the other content that like populates in search for that topic.

Oh,

Travis: yep, yep. And then, You, you mentioned the I b M style guide as like kind of a, a key component for like the, the final editing process, but how do you apply the style guide? Like I B M probably produces a lot of content. So like, is that a manual process with people that are reviewing content and kind of comparing it to the style guide?

Or do you use a tool for that?

Eda: Today, there it is, a ma. Today, it is a manual process, but we're actually looking to bring in another tool to automate that process a little bit more. I don't know if it's taboo to kind of talk about another product on this call. So you'll have to tell me, Travis, but we are kind of looking to bring.

Another tool that's focused on content governance can kind of manage those style guide rules effectively.

Travis: Yeah, I, I, I think it'd be fine to share whatever tool we're looking at using. Yeah, we're always kind of curious,

Eda: we're we're currently exploring acro links as a tool to help us manage content governance overall.

And so it seems like some of their features allow you to. Actually, track some of these individual rules that we might have within our style guide. So it can kind of help us understand. Okay. These are kind of the errors that keep kind of cropping up across our pages so that it's not such a manual process because, you know, humans are not infallible.

So the more that we can kind of use tools to kind of assist folks, I think the, the easier, we'll, we'll kind of make that process overall. Cool.

Travis: And then you also mentioned the importance of fact-checking. Everybody's kind of already heavily focused on fact-checking with the rollout of AI content.

But how do you approach fact-checking at I B M?

Eda: So I think one area like, or one way to kind of fact check is kind of vetting that content through SMEs, but another is looking at kind of claims that people are making and making sure that those claims are substantiated. That's kind of a rule of thumb that we've gotten from legal where.

You know, even within our products, if we're making a claim about our products, we should be able to sus su substantiate that claim through some kind of research whether it's, you know, a t e I report or, you know, a Gartner report. We're, we're not just kind of making things up. And so that is kind of guidance that is also provided to, you know, the inbound team as well.

Where if you're making a claim about specific technology, like, you know, cite your. Sources like refer back to where you're getting that information so people understand, okay, this is like a credible resource to make that type of claim. Cool. Yeah,

Travis: that's helpful. And then Joe just mentioned, he said he had noticed the I B M skyrocket and the search results in the last like six months or so.

I was curious what's the why and like what's the, the, what's driving that, that growth?

Eda: I mean, I think. It's a lot of, you know, foundational like growth. A lot of what we've done over the years, like search, is like a long, like a long game. It's not a short game. And so we've done a lot of great work over the years that just kind of builds upon each other.

Or upon itself. And so it's not that we've done something monumentally different over the last six months. We've maybe gotten some more investment to focus on this within the last six months. But we haven't done something incredibly different from the process that I just showed you.

It's just we, we've been able to kind of build out the team more and then just kind of reap the benefits of, you know, everything that we've done in the past.

Travis: Cool, cool. Yeah. Compounding efforts. And this might be the last question. How do you think about kind of prioritizing new content versus updating or like kind of retiming existing content that may not be producing as it should be or it used to be?

Eda: So I, I think it's important to think about kind of the delta of, like, what you're going to get from that optimization. So if something isn't performing as well, if you did optimize it, would What's the potential traffic uplift that you would get from that optimization versus creating a net new page?

Because if it's like a really big topic or something that's like a heavy hitter, it actually might make sense to focus on that optimization versus creating a net new page on maybe a smaller topic. Because, you know, ranking on. Page one, for, like, machine learning is going to drive an insane amount of traffic for your business.

But you know, maybe ranking on page one for, I don't know, a smaller topic. I'm trying to think of a smaller one off the top of my head. But maybe, like intelligent automation is probably less important because the size of those two topics are, are not the same. But if you wanted to, like, create something, I don't know, net new again, it would just kind of depend on the size of that topic.

So if we were trying to think of something net new now, let's see, anomaly detection is like something that's like net new that we're working on right now. And so, but that one's fairly large, so it's a bad example, but I dunno, I maybe hope, hopefully, you get the, the point is like a smaller topic is going, is going to kind of take should be like evaluated differently if like the optimization that you're trying to make it is larger in size, or it's going to have a larger impact.

Travis: Yeah, I think that makes sense, and I get answers to the question as well. That's all the questions we have. Thanks again, Eda, for joining us today and giving a presentation. And then everyone gives Edda a follow and a shout-out on LinkedIn. Drop it in the chat. And Edda, do you have any last-minute words before we give everyone their day back?

Eda: I don't. Thank you for kind of inviting me to this webinar. Hopefully, it was helpful and insightful. Feel free to reach out via LinkedIn, and apologies for the technical di difficulties at the start of the call. Thanks to everyone for hanging in there with us. Appreciate it. No worries.