This BriefingsDirect IT leadership interview focuses on how Associated Surgeons and Physicians, LLC in Indiana went from zero to 100 percent virtualized infrastructure, and how many compliance and efficiency goals have been met and exceeded as a result.
In part one of a two-part sponsored interview series, we discuss how a mid-market health services provider rapidly adopted server and client virtualization, and how that quickly lead to the ability to move to mobile, bring your own device (BYOD), and ultimately advanced disaster recovery (DR) benefits.
Associated Surgeons and Physicians found the right prescription for allowing users to designate and benefit from their own device choices, while also gaining an ability to better manage sensitive data and to create a data-protection lifecycle approach.
Here to share his story on how they did it, we welcome, Ray Todich, Systems Administrator at Associated Surgeons and Physicians. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions . Disclosure: VMware is a sponsor of BriefingsDirect podcasts.]
Here are some excerpts:
Gardner: When I go to the physician’s office, I see how they've gotten so efficient at moving patients in and out, the scheduling is amazing. Every minute is accounted for. Downtime is just very detrimental and backs up everything. This critical notion of time management is so paramount.
Todich: Oh, it’s absolutely massive. If we have a snag somewhere, or even if our systems are running slow, then everything else runs slow. The ability that virtualization gives us is the core or heart of the entire infrastructure of the business. Without an efficient heart, blood doesn’t move, and we have a bigger problem on our hands.
Gardner: So over the past 10 or 15 years, as you pointed out, technology has just become so much more important to how a health provider operates, how they communicate to the rest of the world in terms of supplies, as well as insurance companies and payers, and so forth. Tell me a little bit about Associated Surgeons and Physicians. How big is the organization, what do you do, how have they been growing?
Todich: Pretty rapidly. Associated Surgeons and Physicians is a group of multi-specialty physicians and practices in Northeast Indiana and Northwest Ohio.
It began at the practice level, and then it really expanded. We're up to, I think, 14 additional locations and/or practices that have joined. We're also using an electronic medical record (EMR) application, given to us by Greenway, and that’s a big one that comes in.
We're growing exponentially. It went from one or two satellite practices that needed to piggyback Greenway, to probably 13 or 14 of them, and this is only the beginning. With that type of growth rate, you have to concern yourself with the amount of money it costs to serve everybody. If you have one physical server that goes out, you affect hundreds of users and thousands of patients, doctors, and whatnot. It’s a big problem, and that’s where virtualization came in strong.
Gardner: How about this in terms of the size of the organization? How many seats are you accommodating in terms of client, and then what is it about an IT approach to an organization such as yours that also makes virtualization a good fit?
Todich: Right now, we have somewhere around 300 employees. As far as how many clients this overall organization has, it’s thousands. We have lots of people who utilize the organization. The reality is that the IT staff here is used in a minimalist approach, which is one thing that I saw as well when I was coming into this.
One or even two persons to manage that many servers can be a nightmare, and on top of that, you try to do your best to help all the users. If you have 300-plus people and their desktops, printers, and so forth, so the overall infrastructure can be pretty intimidating, when you don’t have a lot of people managing it.
Going virtual was a lifesaver. Everything is virtualized. You have a handful of physical ESX hosts that are managing anything, and everything is stored on centralized storage. It makes it considerably efficient as an IT administrator to utilize virtualization.
That’s actually how we went into the adoption of VMware View, because of 300-plus users, and 300-plus desktops. At that point, it can be very hairy. At times, you have to try and divine what the right answer is. You have this important scenario going on, and you have this one and another one, and how do you manage them all. It becomes easier, when you virtualize everything, because you can get to everything very easily and cover everyone’s desktops.
Gardner: What attracted you, at the beginning, to go to much higher total levels of server—and then client—virtualization?
Todich: When I first started here, the company was entirely physical. And as background, I came from a couple of companies that utilized virtualization at very high levels. So I'm very aware of the benefits, as far as administration, and the benefits of overall redundancy and activities—the software and hardware used to allow high performance, high availability, access to people’s data—and still allow security be put in place.
When I came in, it looked like something you might have seen maybe 15 years ago. There were a lot of older technologies in place. The company had a lot of external drives hanging off the servers for backups, and so on.
My first thing to implement was server virtualization, which at the time, was the vSphere 4.1 package. I explained to them what it meant to have centralized storage, what it meant to have ESX host, and how creating virtual machines (VMs) would benefit them considerably over having physical servers in the infrastructure.
I gave them an idea on how nice it is to have alternate redundancy configured correctly, which is very important. When hardware drops out, RAID configuration goes south, or the entire server goes out, you've just lost an entire application—or applications—which, in turn, gives downtime.
I helped them to see the benefits of going virtualized, and at that time, it was solely for the servers.
Gardner: How long did it take you to go from being 100 percent physical to where you are now, basically 100 percent virtual?
Todich: We've been going at it for about about a year-and-a-half. We had to build the infrastructure itself, but we had to migrate all our applications from physical to virtual(P2V). VMware does a wonderful job with its options for using P2V. It’s a time saver as well. For anybody who has to deal with the one that’s building the house itself, it can really be a help.
VMware, in itself, has the ability to reach out as far and wide as you want it to. It’s really up to the people who are building it. It was very rapid, and it’s so much quicker to build servers or desktops, once you get your infrastructure in place.
In the previous process of buying a server, in which you have to get it quoted out and make sure everything is good, do all the front-end sales stuff, and then you have to wait for the hardware to get here. Once it’s here, you have to make sure it’s all here, and then you have to put it altogether and configure everything, so forth. Any administrator out there who's done this understands exactly what that’s all about.
Then you have to configure and get it going, versus, "Oh, you need another server, here, right click, deploy from template," and within 10 minutes you have a new server. That, all by itself, is priceless.
Gardner: And you have a double whammy here, because you're a mid-market size company and don’t have a large, diversified IT staff to draw on. At the same time, you have branch offices and satellites, so you're distributed. To have people physically go to these places is just not practical. What is it about the distributed nature of your company that also makes virtualization and View 5.1 a good approach for a lean IT organization?
Todich: It helped us quite a bit, first and foremost, with the ability to give somebody a desktop, even if they were not physically connected to our network. That takes place a lot here.We have a lot of physicians who may be working inside of another hospital at the time.
Instead of them creating a VPN connection back into our organization, VMware View gave them the ability to have a client on their desktop, whether it be a PC, a MacBook, an iPod, an iPad, or whatever they have, even a phone, if they really want to go that route. They can connect anywhere, at anytime, as long as they have an Internet connection and they have the View client. So that was huge, absolutely huge.
They also have the ability to use PC-over-IP, versus RDP, That’s very big for us as well. It keeps the efficiency and the speed of the machines moving. If you're in somebody else’s hospital, you're bound to whatever network you are attached to there, so it really helps and it doesn’t bother their stuff as much. All you're doing is borrowing their Internet and not anything else.
Gardner: Tell me a bit more about your footprint. We've spoken about vSphere 4.1 and adopting along the path of 5.1. You even mentioned View. What else are you running there to support this impressive capabilities set?
Todich: We moved from vSphere 4.1 to 5.1, and going to VMware View. We use 5.1 there as well. We decided to utilize the networking and security vCloud Networking package, which at the time was a package called vShield. When we bought it, everything changed, nomenclature wise, and some of the products were dispersed, which actually was more to our benefit. We're very excited about that.
As far as our VDI deployment, that gave us the ability to use vShield Endpoint, which takes your anti-virus and offloads it somewhere else on the network, so that your hosts are not burdened with virus scans and updates. That’s a huge gain.
The word huge doesn’t even represent how everybody feels about that going away. It's not going away physically, just going away to another workhorse on the network so that the physicians, medical assistants (MAs), and everybody else isn’t burdened with, "Oh, look, it's updating," or "Look, it's scanning something." It's very efficient.
Gardner: You mentioned the networking part of this, which is crucial when you're going across boundaries and looking for those efficiencies. Tell me a bit more about how the vCloud networking and security issues have been impacted.
Todich: That was another big one for us. Along with the networking and security package comes a portion of the package called the vShield Edge, which will ultimately give us the ability to create our own DMZ the way that we want to create it, something that we don’t have at this time. This is very important to us.
Utilizing the vShield Edge package was fantastic, and yet another layer of security as well. Not only do we have our physical hardware, our guardians at the gate, but we also have another layer, and the way that it works, wrapping itself around each individual ESX host, is absolutely beautiful. You manage it just like you manage firewalls. So it’s very, very important.
Plus, some of the tools that we were going to utilize we felt most comfortable in, as far as security servers for the VDI package, that you want them sitting in a DMZ. So, all around, it really gave us quite a bit to work with, which we're very thankful for.
Gardner: One of the things, of course, that’s key in your field is compliance and there's a lot going on with things like HIPAA, documents, and making sure the electronic capabilities are there for payers and provided. Tell me a bit about compliance and what you've been able to achieve with these advancements in IT?
Todich: With compliance, we've really been able to up our security, which channels straight into HIPAA. Obviously, HIPAA is very concerned with people’s data and keeping it private. So it’s a lot easier to manage all our security in one location.
With VDI, it's been able to do the same. If we need to make any adjustments security wise, it’s simply changing a golden image for our virtual desktop and then resetting everybody's desktops. It’s absolutely beautiful, and the physicians are very excited about it. They seem to really get a hold of what we have done with the ability that we have now, versus the ability we had two years ago. It does wonders.
Upgrading to a virtual infrastructure has helped us considerably in maintaining and increasing meaningful use expectations, with the ability to be virtual and have the redundancy that gives, along with the fact that VMs seem to run a lot more efficiently virtually. We have better ways to collect data, a lot more uptime, and a lot more efficiency, so we can collect more data from our customers.
The more people come through, the more data is collected, the more uptime is there, the more there are no problems, which in turn has considerably helped meeting and exceeding the expectations of what's expected with meaningful use, which was a big deal.
Gardner: I've heard that term "meaningful use" elsewhere. What does that really mean? Is that just the designation that some regulatory organization has, or is that more of a stock-in-trade description?
Todich: My understanding of it, as an IT administrator, is basically the proper collection of people's data and keeping it safe. I know that it has a lot in with our EMR application, and what is collected when our customers interact with us.
Gardner: Are there any milestones or achievements you've been able to make in terms of this adoption, such as behaviors, and then the protection of the documents and privacy data that has perhaps moved you into a different category and allows you to move forward on some of these regulatory designations?
Todich: It's given us the ability to centralize all our data. You have one location, when it comes to backing up and restoring, versus a bunch of individual physical servers. So data retention and protection has really increased quite a bit as far as that goes.
Gardner: How about DR?
Todich: With DR, I think there are a lot of businesses out there that hear that and don’t necessarily take it that seriously, until disaster hits. It’s probably the same thing with people and tornadoes. When they're not really around, you don’t really care. When all of a sudden, a tornado is on top of your house, I bet you care then.
VMware gives you the ability to do DR on a variety of different levels, whether it’s snapshotting, or using Site Recovery Manager, if you have a second data center location. It’s just endless.
One of the most important topics that can be covered in an IT solution is about our data. What happens if it stops or what happens if we lose it? What can we do to get it back, and how fast, because once data stops flowing, money stops flowing as well, and nobody wants that.
It’s important, especially if you're recording people’s private health information. If you lose certain data that’s very important, it’s very damaging across the board. So to be able to retain our data safely is of the highest concern, and VMware allows us to do that.
Also, it’s nice to have the ability to do snapshotting as well. Speaking of servers and whatnot, I'll have to lay it on that one, because in IT, everybody knows that software upgrades come. Sometimes, software upgrades don’t go the way that they're supposed to, whether it’s an EMR application, a time-saving application, or ultrasounds.
If you take a snapshot before the upgrade and run your upgrade on that snapshot, if everything goes great and everybody is satisfied. You can just merge the snapshot with the primary image and you are good to go.
If it doesn’t work out in your favor, you have the ability to delete that snapshot and you're back to where you started from before the migration, which was hopefully a functioning state.
Gardner: Let’s look to the future a bit. It sounds as if with these capabilities and the way that you've been describing DR benefits, you can start to pick and choose data center locations, maybe even thinking about software-defined networking and data center. That then allows you to pick and choose a cloud provider or a hosting model. So are you thinking about being able to pick up and virtually move your entire infrastructure, based on what makes sense to your company over the next say 5 or 10 years?
Todich: That’s exactly right, and the way this is growing, something that's been surfacing a lot in our neck of the woods is the ability to do hosting and provide cloud-based solutions, and VMware is our primary site on that as well.
But, if need be, if we had to migrate our data center from one state to another, we'll have the option to do that, which is very important, and it helps with uptime as well. Stuff happens. I mean, you can be at a data center physically and something happens to a generator that has all the power. All of a sudden, everybody is feeling the pain.
So with the ability to have the Site Recovery, it’s priceless, because it just goes to location B and everybody is still up. You may see a blip or you may not, and nothing is lost. That leaves everybody to deal with the data center issue and everything is still up and going, which is very nice.
Gardner: I imagine too, Ray, that it works both ways. On one hand, you have a burgeoning ecosystem of cloud and hosting, of providers and options, that you can pursue, do your cost benefit analysis, think about the right path, and create redundancy.
At the same time, you probably have physicians or individual, smaller physician practices, that might look to you and say, "Those guys are doing their IT really well. Why don’t we just subscribe to their services or piggyback on their infrastructure?" Do you have any thoughts about becoming, in a sense, an IT services provider within the healthcare field? It expands your role and even increases your efficiency and revenues.
Todich: Yes, our sights are there. As a matter of fact, our heads are being turned in that direction without even trying to, because a lot of people are doing that. It’s a lot easier for smaller practices, instead of buying all the infrastructure and putting it all in place to get everything up, and then maintaining it, we will house it for you. We'll do that.