Technical notes on Radeon HD, straight from AMD engineers. I am mostly documenting this to avoid the bus factor...

ddx == xorg

Radeon HD mode setting:

There are two basic parts we generally work with when programming the display hardware, the crtc and the encoder.
The crtc is basically a display controller and covers the display fb offset, the timing, and the pll setup.
The encoder is the part that takes the bit stream from the crtc and formats it for the proper electrical signal for
the connected monitor (TMDS, LVDS, DAC, DP, etc.).  It's probably easiest to look at the ATOM tables using AtomDis
(more on this below) and follow the same pattern in your driver.  Follow atombios_crtc_mode_set() and radeon_atom_encoder_mode_set()
in the drm.  Basically the modeset sequence looks like:

Disable encoder (dpms off)
Disable crtc (dpms off)
Set up the crtc to encoder routing
Program the pll
Program spread spectrum if applicable
Program mode timing
Program the crtc base address
Program the overscan registers
Program the scaler
Program the encoder
Enable the crtc (dpms on)
Enable the encoder (dpms on)

Note that the pll provides the pixclock for the crtc and the link clocks for the encoders, so it needs to be programmed for all displays.
For displayport on evergreen and newer asics, multiple displays can be driven by the same pll due to the way DP clocking works.

Radeon HD chipset families:

All of these chips have different display hardware so ATOM is your best bet to minimize the changes required to support each generation.

The Display blocks families look like:

Radeon - r1xx-r4xx
AVIVO/DCE 1.0 - r5xx
DCE2.0 - R600, RV610, RV630, RV670, RS690, RS740
DCE3.0 - RV620, RV635, RV770, RS780, RS880
DCE3.2 - RV710, RV730, RV740
DCE4.0 - evergreen (HEMLOCK, JUNIPER, CYPRESS, REDWOOD, CEDAR)
DCE4.1 - fusion (SUMO, SUMO2, PALM)
DCE5.0 - northern islands (BARTS, TURKS, CAICOS, CAYMAN)
DCE6.0 - southern islands (LOMBOK, CAPEVERDE, PITCAIRN, TAHITI)

Each family programs slightly or some cases, majorly differently than others.

A tale of two AtomBIOSes

You can pretty much ignore the atom parser in the ddxes at this point.
That's the internal one we use at AMD that we used for the original open source port.
It's only used for UMS (usermode modsetting).  The one in the kernel was rewritten
to be more portable and easier to follow.  With KMS (kernel modesetting), the ddx
does not touch the hw at all; you can pretty much ignore all the ddx modesetting code.
With KMS, all the ddx does is build command buffers for 2D acceleration, and call ioctls
into the drm to set modes.

AtomBios info

Atom is basically a byte code scripting language used to write little scripts to handle
basic card initialization tasks (asic init, setting engine/memory clocks, modesetting).
There are two sets of tables in ATOM: command tables (basically scripts that execute certain
functionality) and data tables (structs that store board/system specific information (type
and number of connectors/encoders used on the board, power states, ddc lines, panel info, etc.).
The command tables are versioned and there are specific structs defined for the inputs to the
command tables.  These are all defined in atombios.h and ObjectID.h.  Command tables can also
call other command tables and look up data in data tables.

The ATOM command scripting language is pretty simiple.  It's basically an OP, dst, src.
Srcs and dsts can be registers, parameter space, fb space, or workspace (which is basically
a set of special attributes used by the parser).  

I'd suggest dumping a few tables using AtomDis to understand the structure.  A good place to
start is a simple table like EnableCrtc.  Let me know if you have any questions.

Calculating phase locked loops PLL (Phase locked loop) is an electronic device to generate a programmable clock frequency. PLL computations can be *complex*... here is the simplified version which may apply to other video cards:

pixelClock = (pll reference frequency (xtal) * pll feedback divider) + (pll reference frequency (xtal) * pll's feedback divider decimal place ) / (pll's reference divider * post divider)

TRAVIS and NUTMEG external encoders

TRAVIS and NUTMEG are tricky
they are DP to LVDS and DP to VGA bridge chips
so you need to set up the digital encoder and transmitter and the bridge
also, starting with dce3, there is a table called AdjustPixelClock that you need to call to calculate the proper pixel clock
in some cases it will return hardcoded dividers or special pixel clocks
e.g., for clocks less than 60 mhz (IIRC), on dce3+ cards, you need to set the pixel clock to double the requested clock rate.  then there is a special secondary divider that will split it back to the 1/2 rate
TRAVIS and NUTMEG are basically DP, but more complicated
you still have to do link training, etc.

Ontario (PALM, E-350, E-450) has native VGA and LVDS, but some oems use the DP bridges anyway
llano (SUMO, SUMO2) has no native LVDS or VGA so it has to use the bridge chips
  in this case, there will be two encoders associated with the LVDS or VGA connector
  a UNIPHY/1/2 and NUTMEG or TRAVIS
if there is no external encoder, it's just regular LVDS
also, DP uses aux rather than i2c for DDC
the tricky part is, DP is backwards compatible with tmds, so the ddc pads can run in i2c or aux mode depending on what type of monitor is connected
so you have to check if the attached monitor is DP or TMDS, then use aux or i2c to read the EDID