Back when I was running Android 2.3 Gingerbread, there were a lot of apps that I sideloaded to my device. Now with Windows 7 and Windows 8.1, most Android devices can connect to your computer using USB without any additional software.
In Windows XP, devices are not easily connected. Testing with a Samsung Galaxy Nexus, Samsung Galaxy SIII, Nexus 5, and Nexus 7 2013, I found that each device couldn't auto-install the proper drivers and be recognized through Windows Explorer.
My thought was that if I downloaded the general Android drivers, these devices would connect through USB without having to use any additional software. The result was that every device would not install the proper drivers for each device, making them inaccessible in Windows XP so I couldn't drag-and-drop files into the internal memory.
There is a complicated method for pushing APKs but you need to install the Android SDKand Java JDK (Java Development Kit) to use the ddms batch file to access the file explorer to push apk files.
An easier method would be using cloud storage. With Google Drive, you can upload APKs and install them from the app on an Android device. It's now much easier to install APK files using any operating system.

Enable installation of unknown sources

Before you can install an app outside of Google Play, you need to enable the ability to install APKs from Unknown sources. This option is under Security in Settings.
Be aware that installing apps outside of Google Play will be at your own risk. Sometimes these apps don't have full support on your version of Android and there are security risks when installing apps from third party sources.

Upload and install through Google Drive

Since Google Drive can store any digital file you want, you can store different APK files in its own folder for access whenever you want. Open your web browser and sign into Google Drive. Creating a folder will help you keep track of your different APKs.
After creating the file, upload the APK. Google Drive will show a percentage status of the upload. If you're uploading multiple apks, then be patient and wait for the entire upload to complete.
Now that the upload is complete, open Google Drive on Android and locate the same folder in the app. Open the app and tap on the file care in Google Drive. This will prompt Google Drive to open the APK.
Once the APK is finished loading, Android will ask if you want to install the app. The example used is the Google Search APK. As long as you have enabled installations from unknown sources, the APK should install.
Depending on the size of the app, installation time will vary. But once it has completed, you should have that app installed.

Install through Windows XP

Back when I originally wrote how to install APKs through Windows XP, the solution was to use the Android SDK. This method still exists, but I thought two years later it might be possible for Windows XP to recognize Android devices if updated drivers were installed. Unfortunately, I found that there are still problems with Windows XP recognizing devices through the MTP (Media device) connection. Funny enough, I have installed CyanogenMod 10 on one of my personal devices and this was recognized by Windows XP as a removable drive.
But if you don't have Google Drive, then here's the original way to install APKs using the Android SDK.
The tools you're going to need:
The Android SDK requires the Java JDK so it's best to install them both at the same time. Astro File Manager can be installed via Google Play. After installing the prerequisite programs, you are ready to begin.

Android SDK

For non-developers, the Android SDK can be a confusing program. The good thing is that you don't need to worry about most of the included features.
Start the Android SDK and let it run any updates. This will ensure the SDK has the most recent files. After it finishes running, locate the folder where the Android SDK is stored. This may be in your download folder where you installed the SDK package. Locate the "tools" folder which will contain the file you need to run.
Inside the tools folder, locate the "ddms" MS-DOS Batch File and start it. It is the app that's used to access the storage of the device. This is the tool that you will use to push APKs to the device. You can push other types of files as well if you want.
When you start ddms, it will load the Dalvik Debug Monitor. You will know if your device is recognized when it shows up in the name box. It will also show any processes that are happening in the phone. You don't need to worry about any information that's scrolling. Go to "Device" and find "File Explorer" and open it.
Opening the File Explorer will display the device's storage. This is the base level of storage and it's best not to mess with any of these folders. If you want to load the APK to the device, click on the icon with the phone and the forward arrow. This is the command to manually copy files to storage. If you need to create a new folder, press the "+" icon and that will allow you to create a new folder.
Select a folder that you want to copy the apk in. The location isn't too important, you need to be able to locate the file when installed through the device though.
After you copied the file to the device, you can close Dalvik Debug Monitor and the Android SDK.
On the Android device, open Astro File Manager and locate the APK. Tap on it and Astro File Manager will ask if you want to install the APK. Agree to install and the installation will start. This process should work on the majority of apps. The main problem that could happen is that the app you're trying to install isn't supported by your device.
Generally, using this method isn't the fastest way to install APKs on a device now that you can use Google Drive or other file management apps that can use cloud storage. But if you don't want to store APKs in the cloud, then manually copying them through the debug monitor is the way to go.

Cloud storage is the best solution

With Google using Google Play Services to unify the different versions of Android combined with Android 4.4 KitKat's support of older devices, more apps should support older versions of Android. Installing APKs outside of Google Play used to be a quick method to get apps you wanted quicker, but now there are more concerns with security and apps found outside of Google Play may contain malicious code.
If you have older versions of apps in their APKs form and want to install an old version of an app, then this is a great method. But outside of Windows XP, the other Windows operating systems should automatically recognize and mount Android devices  as a drive.

Check out more tips for getting the best from your Android phonehere.