How to have multiple types of controller input using the input manager

So im working on controller input and im wondering how best to impliment controller input which will support multiple controllers.

let me explain, in the input manager you can have a set of inputs for keyboard controls and a set of inputs for joystick controls (key or mouse and joystick axis), the only issue is how do i make it work for all kinds of controllers as im guessing an xbox controller will differ from a wii controller, for example joystick button 5 might be different on an xbox controller to a wii controller, if i have 1 setup for keyboard and 2 setup for joysticks (xbox and wii controllers) how will the game know weather or not its an xbox controller or a wii controller, or is it possible to have one set of joystick controls which will work for both any help would be greatly appreciated