Chapter 5. The Things No One Seems to Talk About

In the preceding section I’ve laid out many of the nuts and bolts of managing a crisis from a library IT perspective, but there are other matters worth mentioning for library IT leaders who have had limited experience in this area. It is not that these things are not practical matters; they are. But they fall somewhere outside the comfortable rubric we might associate with a more staid treatment of the subject. They are the “that’s good to know” sort of personal experience that can forewarn members of the leadership element who have been put in the position of managing such crisis elements for the first time. Some of these will take the form of recommendations from lessons learned. Some of them will just be a heads-up about what you might expect. I believe they are worth covering, and after you read this section I hope you agree.

Put It to Paper

One of the biggest mistakes I made just before the Hale fire was that I was too confident about our wiki. At the time we used Atlassian Confluence, a product that predated my arrival at K-State, and it worked pretty well. We had a lot of very thorough documentation on it, including the departmental emergency contact numbers. As convenient as it was, that fact did me no good at all when the wiki was one of many products knocked offline along with the data center due to the fire. Our department’s head sysadmin had a version of the wiki that the library would use for an internal communications backup in AWS within a few days, but that didn’t help me much when I needed to re-establish communications for the department in the wake of the fire. Luckily I had the numbers of a few of my departmental colleagues in my phone. I used that as the core to work from as I asked them to reach out in turn to anyone whose number they had until we were all on Slack together. I learned a valuable lesson from that. I love digital systems, but even distributed digital systems can fail quickly in a crisis. Have core information, like those emergency contacts, on a good old-fashioned piece of paper. The same goes for any place where, in an emergency, you have a series of steps you need to follow. In my opinion, very little beats a simply worded, straightforward checklist for those situations that has been transcribed onto a heavily laminated piece of paper with a handy grease pencil. If you really want to live right, attach that grease pencil to the checklist with a thick string. That’s the way we did it in the military and it worked.

Finagle’s Law and Disaster Preparedness

Most of us are familiar with that old chestnut: Murphy’s Law. Fewer are familiar with the extrapolation from that: Finagle’s Law. Put simply, Finagle’s Law states that anything that can go wrong will go wrong at the worst possible time. That will be true in a crisis more often than you might imagine until you end up hip-deep in one. As I noted earlier in this report, no more than a few days into the Hale fire crisis, I found myself severely ill and on a course of antibiotics. It was easily one of the worst times in my life for that to happen. During other crises I have had to deal with personnel issues, including the loss of key staff, simultaneous failures of systems not directly related to the main crisis, demands emanating from other parts of campus, poorly timed university initiatives, and a variety of additional problems. In every case these colocated challenges had one important thing in common: the timing of their appearance was terrible. I promise you that my experiences are not an outlier. When trouble knocks on your door, expect it to bring friends, know that they’ll arrive late, and don’t be surprised when a few of them decide to sleep on your couch no matter how badly you try to get them to leave.

In the face of that last, colorful paragraph it may seem like I take a dim view of disaster planning. Far from it. Eisenhower is often quoted as having said that “In preparing for battle I have always found that plans are useless, but planning is indispensable.”1 What the general meant was that, in the moment, Finagle will have its way and plans will need to be continually readjusted to fit the reality of the situation. But through the act of planning, preparations will have been made, alternatives considered, and the apparatus needed to adjust to realities on the ground constructed. Even the act of turning one’s mind toward the problem of planning for a crisis puts the subconscious to work preparing for the worst. Planning takes effort, and engaging in that effort creates an attitude of preparedness.

At K-State Libraries a tremendous amount of general preparation had resulted in a highly organized response plan, well-trained responders, and immediately available response supplies. I recall comments that even the firefighters who put out the fire were impressed. The only area that would have benefitted from more attention was technology. The IT department had not been included in the plans in any meaningful way, which meant in areas like technology redistribution, plans had to be spun up on the fly. Other dimensions to the Hale fire, such as the data center being knocked off line, had not been considered by the disaster planning committee or IT. Nonetheless, thanks to extensive disaster planning and the mindset that it inculcated in library personnel, the initial response by library employees was extremely effective.

At Washington University Libraries, planning for the COVID-19 pandemic began at a very early point. Organized by the university librarian as soon as it became apparent that the COVID pandemic had the potential to affect the US in a broader way, brainstorming sessions began that resulted in the broad outlines of a plan. These sessions also facilitated a shift in perspective by all participants toward considering how the pandemic might affect their individual areas. This meant that planning also began at departmental levels, often informally, as a direct result of the fact that key associate university librarians and department heads were forced to consider contingencies as part of a broader planning process at the organizational level. As the tempo of the crisis increased, brainstorming turned into practical planning between departments. When the requirement to move operations off of campus came, rather abruptly, from Washington University administration, the library already had preliminary plans developed to distribute computing equipment, widen VPN access, and move staff out of the library at a brisk pace.

In these examples we can see that plans had some utility, but the act of planning itself actively changed the mindset of participants. As most managers can attest, given the challenges we must all face each day, it can be difficult to extricate ourselves from more focused mindsets. Yet, as managers, doing exactly that is fundamental. Much of what we must do as leaders is plan and work to see a larger picture. Taking the time to do this, and actually engaging in such an exercise, is legitimate work. And working with colleagues and our own staff to try and plan ahead for challenges creates a mindset that such challenges are inevitable and can be met with planning, flexibility, and grit.

The Reward for Success Will Often Be Greater Challenges

Sometimes, when we confront terrible disasters and monumental challenges with limited resources, we lose sight of the fact that such victories will not bring an end to challenges themselves. In fact, often the reward for success in a crisis will be more challenges from our colleagues. In the case of both of the crises that I have discussed to this point, when new service types were facilitated by our IT departments in response to a crisis, the inevitable response was additional requests for an expansion of those services or the creation of others. This can feel overwhelming . . . especially when you have been stretching staff time and resources to the limit in order to provide help in the first place. Sometimes you will simply need to be clear with your stakeholders about the capacity your team has. Sometimes you will need to explain to stakeholders that you may be able to expand a service eventually, but not immediately. But in every case you need to step back and take a breath. And yes, I’m saying that as if it’s easy, and it isn’t. It’s very hard. Nonetheless, while it may not feel like it at the time, such a response from your colleagues and stakeholders is a sign that you and your team are doing work that matters to them. When you and your team need some breathing room, you should say so. That can feel like you’re failing in some way, but you are not. Teams may work past normal capacity in the early phase of a crisis, but continuing to push in such a fashion is not maintainable. Most of your colleagues will understand that. Even the ones that don’t will benefit from your setting boundaries.

The Danger of an Unqualified “I Don’t Know”

One thing that remains a commonality in confronting tech issues, whether it is under crisis or in more nominal circumstances, is the danger of a naked “I don’t know.” For someone working in IT, giving an initial, unqualified response of “I don’t know” to a problem is very ill advised, no matter how true it might be. This may seem counterintuitive. Most of us have been in positions where we did not receive fully honest responses to a question, and I think most of us would agree that we found such responses less than useful. Most of us have heard, said, or both, that we wished that if someone didn’t know something, they would just admit it.

To be clear, I am not advocating that IT staff and leaders lie to their users. In fact, I would argue that an unqualified “I don’t know” is more disingenuous than a carefully qualified response. Most technologists have some diagnostic ideas when initially confronted with a new problem, even if they don’t think those ideas are the likely cause. More than once I’ve had one of these first-thought responses prove to be correct, despite its low probability of being the culprit. And the truth about technology is that many people find it scary and confusing. We are the experts they call when that technology, for some reason that they find baffling, refuses to work. One sure way to compound their disquiet is to indicate that we are initially baffled ourselves . . . especially in a crisis.

After all, being baffled initially is okay. Given the breadth of technologies many of us work with routinely it is, in fact, inevitable. But we jump in, do the research, talk to the right people, and sort it out. That’s a process we’re familiar and comfortable with, but it provides little comfort to those who want the reassurance provided by the presence of someone who conspicuously knows how to handle a situation.

In technology, when I’m uncertain about the source of a problem, I communicate that to my colleagues, but not in an unqualified manner. While explaining that I will need to look into a matter further (something necessary in the vast majority of cases) I will also describe one or two possible causes for a problem that fit some or all of the conspicuous symptoms I am seeing. In my experience, this has a powerful effect on users and colleagues. It’s one thing to project uncertainty and quite another to explain (much more honestly) that you have a few ideas, but you’re not yet certain what the source of a problem is. Most people understand the latter response and find some reassurance in the fact that they are dealing with a technologist who has enough knowledge and experience that they see some potential causes for a problem at the beginning of the remediation process. Especially under crisis conditions, it is helpful for people to know that there is someone involved in the response that has some possible answers, however preliminary they may be, and they understand that as new information surfaces, understanding of a situation may change. What is not reassuring is to hear the technologist who handles things that, to them, seem rather confusing say that they have no idea what is wrong or what to do about it.

Again, to be clear, I do not lie to my colleagues or my users in these situations. I bring up possibilities that are plausible, even if unlikely, based upon the facts and the circumstances that I am seeing. I work very hard to have more to tell colleagues who are faced with serious conditions than I’m not sure what is wrong or how I can help them.

Strategic Silence

Just as there are many circumstances where it’s necessary to say something, there are others where, no matter how difficult it is, the only healthy way forward is to stay silent. For someone as chatty and as outspoken (for better or for worse) as me, that can be a challenge. But in a leadership position you are often left in the uncomfortable situation of having to pick your battles. I am personally working on improving that skill. And crises have been a very effective, if somewhat abusive, teacher. When dealing with the exhaustion, multitasking, and crushing stress of a crisis situation, it is particularly difficult to avoid being defensive. But surrendering to that impulse isn’t useful.

One situation that epitomizes that fact is one that has occurred in various forms for me across more than one crisis. As I previously mentioned, during times of crisis libraries often increase the frequency of all-staff meetings and increase online access to them. I always try to make myself as available as possible for those meetings, as there are often some requests for information or updates about tech issues. One of the common follow-ups to these questions is the turning of that question toward the rest of the staff. For example, in one meeting, I was asked if we had seen many university network access issues since staff had become remote. I answered that the IT department had not; I knew of only one active ticket to that effect at the time. I noted that there had been some issues with VPN and remote device access early on, but those challenges seemed to have been sorted out. The director then asked my library colleagues to jump in and let her know if they were experiencing issues.

The predictable happened. Around ten people accepted that invitation, expressing discontent with their connectivity to campus systems. I took down names, as did another of my technology colleagues. It sounded bad . . . without any context, it sounded as if I were unaware of significant university technology problems and that a lot of people were suffering with failures that had been unaddressed by my department. Even for those who had some idea of the context involved, it was not a good look.

I did the only thing I could reasonably do in that situation. I gritted my teeth and accepted it for what it was. Our department followed up with every person who spoke out, and what we found was unsurprising. In every case the problems had nothing to do with campus networks or systems . . . those issues, along with connectivity to remote devices and VPN access, had largely been resolved. People were having trouble with their home networking equipment or their internet providers. One person mentioned that their apartment building shared a single Wi-Fi connection, and a large number of people had simultaneously entered lockdown there, causing their connection speed to drop off. Of all of them, only one had a ticket in for IT help, and they had already been informed that the problem was their home router.

It was a frustrating moment, but it was also one that would have only been made worse if I had gotten defensive about the dearth of tickets about connectivity or if I had begun to try and drill down on the remarks by any of the respondents. My library colleagues were justified in letting off a little steam. Some of them had likely misunderstood what was being asked. Others had no idea where the real problem might be and had, for whatever reason, not sought assistance. Some had specifically not sought assistance from the library IT department because they knew how busy we were and were afraid of causing us more work. And no matter what the source of their connectivity problems, it was understandably infuriating for them. I sympathized; they were just trying to do their jobs under new and stressful circumstances. Many of them had home networks that were more than suitable for general connectivity or entertainment and were only now discovering the unexpected frustrations of trying to conduct business over those connections. Local ISP networks were also feeling the strain of elevated traffic at that time. The shabby state of last-mile internet connections in the United States was certainly not my colleagues’ fault. It was a situation where I was going to look like a goat, and I needed to sit back and take it for the good of everyone. If you are a new technology manager, understand that you will have those moments. During a crisis you will have more of them.

I did, however, take a moment when we got to a kudos section of the meeting to call out the excellent work of my library IT colleagues during the crisis. That got at least as enthusiastic a response in the comments area from library staff as the director’s earlier question. As a manager, it’s my job to shoulder criticism at times, but my departmental staff should get to feel the appreciation they’ve earned. Such small gestures go a long way toward helping to bolster morale.

In yet another situation, I was in a planning meeting at an early phase of crisis response where we had just begun to issue temporary laptops for remote work. A schedule had been drawn up and provided to the planning committee and managers for people to collect equipment. However, one department had ignored that schedule, and many of its staff had simply shown up at random intervals on the first issuance day expecting to receive equipment. This was a problem, as the devices required a certain amount of setup. We had streamlined that setup process, but the scheduling had been designed to provide for adequate setup times with some opportunity for IT staff to take breaks and perform prep work in between sessions. I brought this problem up with the head of the responsible department at the next planning meeting and was met with anger. They felt they needed their machines as soon as possible. They were uninterested in the agreed-upon schedule. Unexpectedly, the issue was taken up by a much higher level administrator who sided with them. Despite the fact that no one on the committee had raised any objection to the schedule before, they were adamant that the department in question just get what it wanted.

These were not unreasonable people, despite their reaction. The higher-level administrator, in particular, was one I had a great deal of respect for. However, things were happening quickly, and I suspect they were increasingly concerned that the libraries would not be able to issue equipment by the drop-dead date we had to meet, schedule or no. That concern fueled their reactions.

At that point I ceased engaging on the problem, sat back, and absorbed their rather strident remarks. There was nothing I could say that was going to resolve the situation favorably for my team, and I was dealing with a level of organizational authority that vastly outstripped my own. Furthermore, the argument threatened to derail other important response work that needed to be addressed. It was an extraordinarily stressful moment for me, but continuing to plead my case would have achieved nothing. No one was listening. It was another moment where I needed to simply accept a bad situation for what it was.

Fortunately, the noncompliant staff were mostly confined to a single department. The schedule worked as intended, and everyone who needed their equipment during that issuance period received it. Library IT staff worked incredibly hard to image and issue a large number of laptops in a short amount of time, enabling our colleagues to work offsite.

These situations were stressful, but the important thing to remember is that, fundamentally, everyone was trying to do the right thing . . . they just didn’t happen to agree on what that right thing was. And these were situations where, past a certain point, the only useful course of action that remained was to sit back and take it.

Ideas Take a Team

In any crisis response, teamwork is vital. The IT department must work as a team to coordinate its actions, leverage internal expertise, confront problems using diverse viewpoints, and apply effort effectively. When leveraging that internal expertise, the overall intellectual capital that your team possesses must not be ignored. In responding to any crisis, ideas take a team. When the fire happened at Hale, I had some good ideas about how to respond. But so did every other member of the Technology Services department. Our head sysadmin leaped into action to save critical systems. She needed no direction to do that. She was the one who had the idea to stand up a copy of the organizational wiki for use in early communication efforts before the university got its other systems back online. The coordinator of the LIST unit and her staff put together the assembly-line style process for imaging over a hundred new computers in only two days, and it was one of our developers who immediately jumped in with the idea that we should all gather together and make it a group effort. These are only a few examples of outstanding individuals adding their ideas in an environment that had been made friendly to open contributions by everyone. Together, as a team, we were far smarter in our response than we ever would have been if such contributions had been discouraged and every idea and specific of our response efforts had originated with me or those I reported to. Skilled, intelligent individuals exercising initiative and contributing to our departmental response planning was one of the keys to our success in dealing with a huge, unexpected calamity.

The same was true at Washington University Libraries when the COVID pandemic hit. I was a relatively new manager at the time, having occupied my position for only a few months before we were required to stop reporting to campus. The crisis struck at the most inopportune of times. We had just finished migrating our entire digital infrastructure off of the local data center and were still dealing with cleanup from that operation. We had just lost one of our most experienced and talented staff members. We were at a historically low staffing ebb with multiple projects about to be initiated that would be contemporaneous with the pandemic conditions. Despite these facts, every member of the Library Technology Services department stepped up to the challenges of the crisis. My LTS colleagues worked together to deal with everything from equipment issues to bug fixes on legacy products. Elements of the response were discussed and brainstormed in small groups and departmental meetings. At every turn our plan of action was shaped by ideas and input from across the department.

As a manager in such situations, it was my job to make decisions and provide overall direction rather than to try to be the source of all good ideas about how to respond. In fact, a big part of managing a department, in my opinion, is about encouraging contributions and creating a forum to express ideas, concerns, and viewpoints, even if I don’t agree with them. Empowered people are capable of outstanding efforts. They feel a sense of ownership over their job and the organization they are a part of. They feel empowered to express themselves if they work for someone that they know will take them seriously and who is capable of being persuaded by facts, data, and good ideas.

There is something quietly sacred, to me, about being entrusted with the intellectual production of others. Part of that comes from my reverence for good ideas, but another part stems from the trust that goes along with such a collaborative process. I have worked for a few supervisors, some placed quite high in organizations, who could not be trusted with ideas. The worst of them distrusted any ideas that were not their own. They swept them aside, or pretended that the ideas were theirs, sometimes in the face of extraordinary evidence to the contrary. One of the worst library administrators I have ever dealt with once met with me and two members of a committee I was chairing shortly after they had assumed their position to tell us that our committee needed to start engaging with faculty on the rest of campus. We explained that we had been doing that very thing for months. Those engagement meetings had been documented. The schedule for future engagement meetings was on our Teams site. The administrator’s response was that we had our viewpoint and they had theirs, and they summarily dismissed any further contention throughout that meeting that campus engagement had not been their original idea. They expressed that they were correcting the trajectory of our work. We three left the discussion stunned and discouraged.

Managers should respect the ideas of their staff. They should go to pains to note that such ideas, when passed along to their reporting chains or committees, were not theirs, but rather cite their sources. They should regard the input of their teams as a precious commodity, a resource that is key to success in everything from crisis response to future scaling a department. That is not to say that they must adopt every suggestion, agree with every idea, or refuse to challenge any contribution. Some discussion and disagreement is a healthy feature of an exchange of ideas. And, as the manager, it is your job to decide which ideas to implement and how they should be implemented. But your departmental colleagues should know that they are free and encouraged to express their suggestions about the work, no matter who agrees with them. No one on my team ever got a black mark next to their name just because they disagreed with me. In fact, a common refrain from me is that if one of my ideas is a bad one, I’d much rather hear about it from my team than be left to deal with the repercussions after it’s been implemented.

Managers must build trust with their IT colleagues by properly apportioning credit for good ideas and good work. Dwight D. Eisenhower, always a fine source for quotes, had this to say about the matter: “Leadership consists of nothing but taking responsibility for everything that goes wrong and giving your subordinates credit for everything that goes well.”2 Remember that a team, however ineffective it might sometimes be, can exist without a manager. But a manager is nothing without their team. It can take time to build that trust, but when a team gets there, the results are well worth the investment.

Budgets

One thing that is often pushed to the wayside at the beginning of a crisis is the question of budgets. For some it may even seem crass to talk about budgets at the beginning of a crisis situation. After all, we librarians are entrusted with valuable intellectual and cultural treasures. When any part of those resources is threatened, or any part of that mission is interrupted, it is our job to get things back to one hundred percent for the sake of our patrons, donors, and trustees. At such times it’s difficult to think about what that recovery will cost, and it is unavoidable that the organization will absorb expenses it did not expect. However, I offer this word of advice: never stop thinking about where the money is coming from. In the beginning even administrators are often willing to throw the budget to the winds, but that bill will come due. It always comes due. Remembering that on the front end goes a long way toward making things easier on the back end of a crisis.

One of the challenges to the budget of a crisis is that, by definition, the circumstances of a crisis are special ones. It’s easy for people to feel that the normal rules don’t apply. During one crisis period, a team member suggested that we purchase a piece of equipment that we had been unable to get under ordinary circumstances because purchases were receiving a conspicuously lower amount of scrutiny. I refused that notion on general principles, but it made for a good teachable moment. I had been through smaller-scale crises before and told them the truth of the matter. Even if I agreed with the ethics of that action, and I didn’t, someone always belatedly scrutinizes the books. We might get away with something like that in the short term, but eventually (and rightfully) such unusual purchases would be questioned.

Another budgetary challenge of crisis conditions is helping people understand the trade-off of temporary expenses for recovery expenses. As one example, I have seen departments in crisis make requests to relocate what would have essentially been decorative equipment to temporary service point locations. These were large pieces of ornamentation, not required for the mission of the service point, which would have cost thousands of dollars to move there and back—thousands that would then have to be subtracted from the funds available for permanent recovery efforts. A very poor trade-off.

Inevitably, as situations like remote operations drag out, staff will begin to request additional peripherals and equipment. Some of those suggestions will be practical, and some will be extravagant. It is important for IT departments to take steps to mitigate the more extreme requests and to put any consequent purchases on the firmest budgetary footing possible. One way to do this is to select standard, reasonably priced products in common request categories that are approved for purchase. Staff who want something more lavish should purchase it as personal gear.

It is also important to properly route requests that come into IT as ergonomic in nature. Most libraries have staff who are trained in conducting ergonomic assessments and making organizationally approved adjustments. HR and those staff should meet with the affected employee to generate recommendations for equipment that satisfies the needs of that employee. Simply ordering equipment suggested by affected employees can often lead to expensive acquisitions that don’t solve the problem they were intended to ameliorate. This situation becomes especially tricky when dealing with remote work situations. Libraries should establish clear policies about what workspace environments they are responsible for.

Effort may also need to be put into determining what fund to draw some expenses from. As the manager responsible for most tech fund purchases at Washington University Libraries, I have had to make determinations about what constitutes a true technology purchase and what is more logically situated as a furniture or office supply purchase, which would move it under the aegis of departmental funds.

Notes

1. “Dwight D. Eisenhower 1890–1969, American Republican statesman, 34th President 1953–61,” Oxford Essential Quotations, Fourth ed., edited by Susan Ratcliffe (New York: Oxford University Press, 2016), www.oxfordreference.com/view/10.1093/acref/9780191826719.001.0001/q-oro-ed4-00004005.
2. Originally quoted in Edgar F. Puryear Jr., Nineteen Stars: A Study in Military Character and Leadership (Coiner Publications, 1971), 289.