TLDR Version: talent tree is at the bottom. Go look at it, but no posting in this thread unless you have learned to read.
I found some notes I put together a year ago, and figured I would actually do something with them. Below is my version of what a Shadow Hunter class could look like in WoW. It's just a concept, so I would be interested in feedback from people regarding the talent trees and theorycrafting. This is actually rather detailed, due to the inclusion of reasoning behind decisions, so don't be in a hurry.
These reclusive, wily jungle trolls are considered to be the highest authority within there respective warbands. They are masters of voodoo magics who can use their spirit powers to both heal their allies and place curses upon their hapless enemies. The Shadow Hunters walk the line between darkness and light in hopes of salvaging the future for their savage brethren.
The trolls draw power for their voodoo magics by performing rituals and invoking the Prima Gods, Old Gods, forest spirits, and ancestral spirits: their Loa.
Loa spirits are more powerful than the elements, but not as powerful as gods. Supposedly, these spirits are mighty entities that grant the faithful extraordinary powers. By calling upon these voodoo spirits, the Shadow Hunter gains special blessings with which he can combat darkness and help those in need. The abilities vary according to the Loa the Shadow Hunter calls upon.
The Shadow Hunters faith and communion with the Loa gods grants them shadow and healing magic, and other abilities. Shadow Hunters learn magic that hinders enemies and aids allies, gaining a unique mix of spells that delve into the more occult aspects of the Loa.
Since entirely new, non-existent classes are difficult to manifest at this point, influences for a hybridization comes from mixing elements of existing classes. Also, since the last hero class was a tank / melee dps class, for balance a healer / range dps class would be applicable.
Death Knight - paladin / warlock class combo, tank / melee dps role combo
Shadow Hunter - shaman / warlock / priest class combo, healer / range dps role combo
Tank (4) - Druid / Warrior / Paladin / Death Knight
Melee DPS (6) - Druid / Rogue / Shaman / Warrior / Paladin / Death Knight
Range DPS (7) - Mage / Priest / Warlock / Druid / Shadow Hunter / Shaman / Hunter
Healer (5) - Priest / Druid / Shadow Hunter / Shaman / Paladin
For armor allocation, and gear itemization, maintaining a balance in loot is important. Shadow Hunters would share loot primarily with druids (both balance and resto). While Shadow Hunters could have gone with mail for balance reasons, lore indicates a lighter armored class, and it was important to denote more of a distinction between Shaman / Shadow Hunter since there are already natural similarities.
Plate (3) - Warrior / Paladin / Death Knight
Mail (2) - Shaman / Hunter
Leather (3) - Rogue / Druid / Shadow Hunter
Cloth (3) - Mage / Warlock / Priest
Physical damage is largely divided by agility / strength stats, since the classes can essentially use almost all of the same weapons. For casters, the stats are the same (albeit spirit vs. hit for healer vs. dps) so the distinction comes from weapon proficiency.
Dagger / Sword - Mage / Warlock
Dagger / Mace - Priest / Druid / Shaman
For slight lore reasons but more balance issues, Shadow Hunters would be:
Dagger / Sword / Fist / Staff
Range - Fetish / Thrown
Faction and Race:
From a lore standpoint, Shadow Hunters are trolls. In the interest of faction balance (and to avoid the Shaman / Paladin issue from vanilla) an Alliance race is needed. Again, from a lore standpoint, if Night Elves are closely linked to trolls then it could be a relatively easy argument to introduce primal Night Elves. It actually makes a lot more sense than Draenei Shaman. But, as someone pointed out, it would help if Goblins and Gnomes (ewww) were in the mix, and that wouldn't hurt anything.
While Shadow Hunters could end up going the Death Knight route, I personally disagree with "everything for everyone". I made this all because I liked lore, so I'm going with lore. Death Knights could have been Orc, Undead, Human, Dwarf and worked out fine with classic lore, but the class was ret-con'd.
As far as caster damage is concerned, classes are typically balanced with two elemental schools. While some classes can cast more (ie Shaman with Frost Shock) it isn't necessarily a large part of their regular rotation. Below is a comparison to show how Shadow Hunters retain a unique flavor, and how they fit in with the other classes.
Mage - fire / frost / arcane
Priest - holy / shadow
Warlock - fire / shadow
Druid - nature / arcane
Shadow Hunter - nature / shadow
Shaman - nature / fire
Hybrids have a few options. They can be dual spec (Priest, Warrior, Death Knight), tri spec (Shaman, Paladin), or quad spec (Druid). In this case dual spec was chosen, so if one talent tree is devoted to damage and one tree is devoted to healing, what is done with the third tree? While Shadow Hunters are a cross between Shaman and Warlock for lore and ability, the talent trees are structured more along the lines of a Priest.
Voodoo - damage tree
Superstition - utility / support tree
Mysticism - healing tree
Blizzard is attempting to get away from PvE vs PvP trees, so the trees are structured around choosing a role first (dps or healer), and then using the support tree to determine how offensive or defensive your play style is. The Superstition tree was the most difficult to work out, as it needed to be viable for both roles in PvE (at least early in the tree) instead of completely defaulting to PvP.
I went with the basic premise of small heal, big heal, HoT, group heal, shield. Wards and Hexes are meant to add a little flavor, but to make this classes unique from the other healers (and possibly more desirable) was to find ways of weaving in damage. Instead of a healer that can damage, creating the healer that should damage. Examples are [Empowered Malaria], [Voodoo Link] and [Voodoo Siphon]. [Spirit Link] (Spirit Walker, WC3 TFT) was originally going to be given to Shaman in WotLK (replaced with Riptide), but was probably going to be too difficult to balance. [Voodoo Link] takes that idea, but is similar to a Warlock's [Health Funnel] and the green drakes in Occulus / Eye of Eternity, so the Shadow Hunter has to damage the enemy to heal themselves, and subsequently pass that healing to tanks. This is also a mechanic used in Warhammer.
Casting rotations have gotten more interesting since earlier expansions of WoW. Not all of them are necessarily complicated, but the intent is to add some variables to the mix. For example in TBC elemental Shaman cast Lightning Bolt. In WotLK, the rotation became Flame Shock + Lava Burst + Lightning Bolt + Chain Lightning depending on haste values due to the cooldown of Lava Burst. Destruction Warlocks spammed Shadowbolt and owned the meters in TBC (until mid-Sunwell and Incinerate). In WotLK, Immolate + Chaos Bolt + Conflagarate + Incinerate. For Druids, talents encouraged weaving Wrath and Starfire based on crits and procs.
While every class has their spamable ability, there also needs to be a rotation and situational spells. The big thing that helped rotations was priority due to cooldowns, and buffs that reinforced other abilities. (Flame Shock enabling Lava Burst to crit, Immolate being required for Conflagarate to be cast, etc.)
The spamable spell is [Black Voodoo] and have a volley animation.
Additional spells are: [Malaria] instant DoT, [Fel Strike], [Stormspear], [Burning Heart] Holy Fire animation - but blue, [Meuh'zala], and then concoctions for AoE.
I'm curious to see what theorycrafting people come up with regarding rotations, and adjustments / improvements will be made based on input.
This was definitely the more challenging tree to design. It was intended to support the other two trees as far as PvE and PvP while healing or damaging. Additionally it works in its own right as a heavy PvP / questing tree with a greater focus on control, but still viable for say running dungeons and heroics.