Trying to Make a Topdown, 3rd Person Ladder

So I'm basically making a SNES Zelda clone and am having problems making a ladder that works.

All of the examples I've found bind up and down movement to either the direction of the mouse or a certain keyboard key, which doesn't work in top-down magical view an one can climb a ladder in any of four directions both up AND down.

The very closest I got to anything similar was taking this ladder and applying my camera to it, but to no avail.

Anyway, if anyone has any kind of clue where to look or what to do I'd love it!

Special Note: I'm not new to programming in general, but a complete newbie to Unity coding (though I'm going through tutorials to learn better), so if you can be as specific as possible it'd be über helpful!