Technical notes on Radeon HD, straight from AMD engineers. I am mostly documenting this to avoid the bus factor...

Developer links. ​Radeon documents etc

ddx == xorg

Radeon HD mode setting:

There are two basic parts we generally work with when programming the display hardware, the crtc and the encoder.
The crtc is basically a display controller and covers the display fb offset, the timing, and the pll setup.
The encoder is the part that takes the bit stream from the crtc and formats it for the proper electrical signal for
the connected monitor (TMDS, LVDS, DAC, DP, etc.).  It's probably easiest to look at the ATOM tables using AtomDis
(more on this below) and follow the same pattern in your driver.  Follow atombios_crtc_mode_set() and radeon_atom_encoder_mode_set()
in the drm.  Basically the modeset sequence looks like:

Disable encoder (dpms off)
Disable crtc (dpms off)
Set up the crtc to encoder routing
Program the pll
Program spread spectrum if applicable
Program mode timing
Program the crtc base address
Program the overscan registers
Program the scaler
Program the encoder
Enable the crtc (dpms on)
Enable the encoder (dpms on)

Note that the pll provides the pixclock for the crtc and the link clocks for the encoders, so it needs to be programmed for all displays.
For displayport on evergreen and newer asics, multiple displays can be driven by the same pll due to the way DP clocking works.

Radeon HD chipset families:

All of these chips have different display hardware so ATOM is your best bet to minimize the changes required to support each generation.

The Display blocks families look like:

Radeon - r1xx-r4xx
AVIVO/DCE 1.0 - r5xx
DCE2.0 - R600, RV610, RV630, RV670, RS690, RS740
DCE3.0 - RV620, RV635, RV770, RS780, RS880
DCE3.2 - RV710, RV730, RV740
DCE4.0 - evergreen (HEMLOCK, JUNIPER, CYPRESS, REDWOOD, CEDAR)
DCE4.1 - fusion (SUMO, SUMO2, PALM)
DCE5.0 - northern islands (BARTS, TURKS, CAICOS, CAYMAN)
DCE6.0 - southern islands (LOMBOK, CAPEVERDE, PITCAIRN, TAHITI)
DCE6.1 - fusion (ARUBA)
DCE6.4 - southern islands (OLAND) (7000 rebranded low-end 8000)
NODCE? - sea islands (HAINAN) (no display hardware, only 3d engine :'( )
DCE8.0 - sea islands (BONAIRE, CURACAO)

Each family programs slightly or some cases, majorly differently than others.

A tale of two AtomBIOSes

You can pretty much ignore the atom parser in the ddxes at this point.
That's the internal one we use at AMD that we used for the original open source port.
It's only used for UMS (usermode modsetting).  The one in the kernel was rewritten
to be more portable and easier to follow.  With KMS (kernel modesetting), the ddx
does not touch the hw at all; you can pretty much ignore all the ddx modesetting code.
With KMS, all the ddx does is build command buffers for 2D acceleration, and call ioctls
into the drm to set modes.

AtomBios info

Atom is basically a byte code scripting language used to write little scripts to handle
basic card initialization tasks (asic init, setting engine/memory clocks, modesetting).
There are two sets of tables in ATOM: command tables (basically scripts that execute certain
functionality) and data tables (structs that store board/system specific information (type
and number of connectors/encoders used on the board, power states, ddc lines, panel info, etc.).
The command tables are versioned and there are specific structs defined for the inputs to the
command tables.  These are all defined in atombios.h and ObjectID.h.  Command tables can also
call other command tables and look up data in data tables.

The ATOM command scripting language is pretty simiple.  It's basically an OP, dst, src.
Srcs and dsts can be registers, parameter space, fb space, or workspace (which is basically
a set of special attributes used by the parser).  

I'd suggest dumping a few tables using AtomDis to understand the structure.  A good place to
start is a simple table like EnableCrtc.  Let me know if you have any questions.

Assigning phase locked loops Radeon HD cards can have several PLL clock sources. Some PLL sources can only be used for certain encoders.

• DCE3.x: DP == PLL1 or PLL2, non-DP == PLL1 or PLL2
• DCE4.0: DP == EXT_PLL or PLL1 or PLL2, non-DP == PLL1 or PLL2
• DCE4.1: DP == PLL1 or PLL2, non-DP == PLL1 or PLL2
• DCE5.0: DP == DCPLL or PLL1 or PLL2, non-DP == PLL1 or PLL2
• DCE6.0: DP == PLL0 or PLL1 or PLL2, non-DP == PLL1 or PLL2
  • I think you might be able to use PLL0 as an independent PLL on DCE6.0 as well, but I don't think it was validated in hw so it may not work correctly depends on the DCE family
• DCE6.1: DP == PLL0 or PLL1 (PHY BCDEF) or PLL2 (PHY A), non-DP == PLL0 or PLL1 (PHY BCDEF) or PLL2 (PHY A)

Calculating phase locked loops PLL (Phase locked loop) is an electronic device to generate a programmable clock frequency. PLL computations can be *complex*... here is the simplified version which may apply to other video cards:

pixelClock = (pll reference frequency (xtal) * pll feedback divider) + (pll reference frequency (xtal) * pll's feedback divider decimal place ) / (pll's reference divider * post divider)

TRAVIS and NUTMEG external encoders

TRAVIS and NUTMEG are tricky
they are DP to LVDS and DP to VGA bridge chips
so you need to set up the digital encoder and transmitter and the bridge
also, starting with dce3, there is a table called AdjustPixelClock that you need to call to calculate the proper pixel clock
in some cases it will return hardcoded dividers or special pixel clocks
e.g., for clocks less than 60 mhz (IIRC), on dce3+ cards, you need to set the pixel clock to double the requested clock rate.  then there is a special secondary divider that will split it back to the 1/2 rate
TRAVIS and NUTMEG are basically DP, but more complicated
you still have to do link training, etc.

Ontario (PALM, E-350, E-450) has native VGA and LVDS, but some oems use the DP bridges anyway
llano (SUMO, SUMO2) has no native LVDS or VGA so it has to use the bridge chips
  in this case, there will be two encoders associated with the LVDS or VGA connector
  a UNIPHY/1/2 and NUTMEG or TRAVIS
if there is no external encoder, it's just regular LVDS
also, DP uses aux rather than i2c for DDC
the tricky part is, DP is backwards compatible with tmds, so the ddc pads can run in i2c or aux mode depending on what type of monitor is connected
so you have to check if the attached monitor is DP or TMDS, then use aux or i2c to read the EDID