I’ve been waiting for OpenGL 3D in VMware for Linux guests for a while now and I’m happy to say I finally have it working. This guide has been specifically tested for the following setup:

• Windows 7 Professional x64 host running VMware Workstation 8
• Kubuntu 11.10, 64bit guest

This guide is based on the original post by khogan, 3D hardware acceleration in Linux. Hello GNOME 3, and the guide by Joost, VMware Linux guest with 3D accelerated graphics.

• This seems to work out of the box on Kubuntu 12.04 Alpha 2, 64bit! Simply update your packages and install VMware tools. (I needed to run a package update twice; the second time included the new mesa drivers. Also, you may want to reboot after each step just to be safe.)
• OpenGL 3D acceleration works out of the box on Kubuntu 12.04!

I recommend making snapshots at various points in case you mess up and want to start over, or if you need to try a different approach if this doesn’t work for you. Also, I’m not responsible for anything you do while following this guide.

Preparing to build

First make sure

1. VMware Tools is installed on the guest
2. “Accelerate 3D graphics” is checked under “Virtual Machine Settings > Display”

Now we need to install the build dependencies:

sudo apt-get install git build-essential autoconf libtool flex bison xorg-dev xutils-dev libx11-xcb-dev libxcb-glx0-dev python-libxml2

Next, make a working directory and download the source repositories:

mkdir ~/vmware-3d && cd ~/vmware-3d
git clone git://anongit.freedesktop.org/git/mesa/drm
git clone git://anongit.freedesktop.org/git/mesa/mesa
git clone git://anongit.freedesktop.org/git/xorg/driver/xf86-video-vmware
git clone git://anongit.freedesktop.org/git/mesa/vmwgfx

Update VMware Tools

Make sure you install the latest VMware Tools and reboot before continuing.

Building and installing the drivers

Go to the working directory:

cd ~/vmware-3d

First we must install drm:

cd drm
./autogen.sh --prefix=/usr --enable-vmwgfx-experimental-api
make
sudo make install
cd ..

Next is mesa:

cd mesa
./autogen.sh --prefix=/usr --with-gallium-drivers=svga --with-dri-drivers= --enable-xa
make
sudo make install
cd ..

Then the drivers:

cd xf86-video-vmware
./autogen.sh --prefix=/usr
make
sudo make install
cd ..

Finally the kernel module:

cd vmwgfx
make
sudo make install
cd ..

I rebooted at this point. Things may look a little strange when it boots, be we are about to fix that.

Updating the kernel

Get back to the working directory:

cd ~/vmware-3d

We also need an updated kernel, so we download

wget "http://kernel.ubuntu.com/~kernel-ppa/mainline/drm-next/2012-01-11-precise/linux-headers-3.2.0-996_3.2.0-996.201201110405_all.deb"
wget "http://kernel.ubuntu.com/~kernel-ppa/mainline/drm-next/2012-01-11-precise/linux-headers-3.2.0-996-generic_3.2.0-996.201201110405_amd64.deb"
wget "http://kernel.ubuntu.com/~kernel-ppa/mainline/drm-next/2012-01-11-precise/linux-image-3.2.0-996-generic_3.2.0-996.201201110405_amd64.deb"

and install them with

sudo dpkg -i linux-headers-3.2.0-996_3.2.0-996.201201110405_all.deb
sudo dpkg -i linux-headers-3.2.0-996-generic_3.2.0-996.201201110405_amd64.deb
sudo dpkg -i linux-image-3.2.0-996-generic_3.2.0-996.201201110405_amd64.deb

Almost done!

Reboot the VM; you should finally have 3D acceleration. When I got it working the application fonts were huge. (As a workaround you can set “Force fonts DPI” under “System Settings > Application Appearance > Fonts”. Reinstalling VMware Tools might also fix the problem; see the last section.)

To check if it’s working, you can install mesa-utils:

sudo apt-get install mesa-utils

and run

glxinfo | less

to look for something like

OpenGL vendor string: VMware, Inc.
OpenGL renderer string: Gallium 0.4 on SVGA3D; build: RELEASE;
OpenGL version string: 2.1 Mesa 8.0-devel (git-e9bcf4d)

You can delete the working directory ~/vmware-3d when your satisfied its working.

If this guide was helpful or if you have any questions, please leave a comment. Enjoy your GPU accelerated VM!

VMware Tools

You will want to reinstall VMware Tools at this point, but there is a good chance it will fail and you will be stuck without it, so make a snapshot before trying. You will need gcc 4.4 before you run the install script:

sudo apt-get install gcc-4.4
sudo vmware-config-tools.pl

Note that I could not finishing reinstalling it becasue it would hang near the last step on

Creating a new initrd boot image for the kernel.
update-initramfs: Generating /boot/initrd.img-3.2.0-996-generic

After (if) it installs reboot the VM. If it doesn’t work we will most likely have to wait for VMware to release an update that works with the new kernel.