index : renesas/drm.git	android live master
	libdrm, cloned from git://anongit.freedesktop.org/mesa/drm	Laurent Pinchart

summaryrefslogtreecommitdiff

log msg author committer range

path: root/shared-core/nouveau_reg.h

Age	Commit message (Collapse)	Author
2008-07-01	NV50: some i2c cleanup	Maarten Maathuis
2008-06-22	NV50: Initial import of kernel modesetting.	Maarten Maathuis
2008-04-05	nv50: primitive i2c interrupt handler	Maarten Maathuis
2008-04-03	nv50: primitive display interrupt handler.	Maarten Maathuis
2008-02-16	nv40: actually init all tile regs.	Ben Skeggs
2008-02-02	nouveau: NV40 can/should now be able to run after the blob.	Maarten Maathuis
	- Moved the fix from the ddx to drm, because it seemed more appropriate. - Don't be shy, report if it works for you or not.
2008-01-04	[PATCH] nouveau: reset AGP on init for < nv40	Stuart Bennett
	This is necessary for AGP to work after running bios init scripts on nv3x, and is seen in mmio traces of all cards (nv04-nv4x) I'm not making the equivalent change to nv40_mc.c, as early cards (6200, 6800gt) use the 0x000018XX PBUS and later cards use the 0x000880XX PBUS and I don't know the effects of using the wrong one
2007-11-14	nouveau: Also wait until CACHE1 gets emptied.	Ben Skeggs
2007-11-14	nouveau: store user control reg offsets in channel struct	Ben Skeggs
2007-11-14	nouveau: funcs to determine active channel on PFIFO.	Ben Skeggs
2007-11-05	drm: remove lots of spurious whitespace.	Dave Airlie
	Kernel "cleanfile" script run.
2007-10-10	nouveau : nv10 and nv04 PGRAPH_NSTATUS are different	Matthieu Castet
2007-10-10	nouveau: try to fix big endian.	Stephane Marchesin
2007-09-30	nouveau : nv20_graph replace nouveau_graph_wait_idle by nouveau_wait_for_idle	Matthieu Castet
	Also clean PGRAPH_CHANNEL macros
2007-09-10	nouveau: nv10: add combiner registers	Patrice Mandin
2007-08-26	nouveau : add NV04_PGRAPH_TRAPPED_ADDR definition	Matthieu Castet
	- fix offset for nv04 - use it in nv10 graph ctx switch for getting next channel - dump NV10_PGRAPH_TRAPPED_DATA_HIGH on nv10+
2007-08-22	nouveau/nv40: Dump extra info on ucode state if ctx switch fails.	Ben Skeggs
2007-08-10	nouveau/nv50: demagic instmem setup.	Ben Skeggs
2007-07-18	nouveau: Add bitfield names for NSOURCE and NSTATUS.	Pekka Paalanen
	Name strings and pretty-printing in nouveau_graph_dump_trap_info().
2007-07-18	nouveau: Replace 0x00400104 and 0x00400108 with names.	Pekka Paalanen
	NV03_PGRAPH_NSTATUS and NV03_PGRAPH_NSOURCE. The prefix NV03 is chosen because nv10reg.h had no versioned prefix, and the code using these registers does not check card_type.
2007-07-11	Added support for PCIGART for PCI(E) cards. Bumped DRM interface patchlevel.	Arthur Huillet
2007-07-09	nouveau/nv50: Initial channel/object support	Ben Skeggs
	Should be OK on G84 for a single channel, multiple channels *almost* work. Untested on G80.
2007-06-28	nouveau: name some regs	Ben Skeggs
2007-06-24	nouveau: NV04 PFIFO engtab functions	Ben Skeggs
2007-06-24	nouveau: NV4X PFIFO engtab functions	Ben Skeggs
2007-04-06	nouveau: make a note about a bit that breaks some cards	Ben Skeggs
2007-03-26	nouveau: move card initialisation into the drm	Ben Skeggs
	The PGRAPH init for the various cards will need cleaning up at some point, a lot of the values written there are per-context state left over from the all the hardcoding done in the ddx. It's possible some cards get broken by this commit, let me know. Tested on: NV5, NV18, NV28, NV35, NV40, NV4E
2007-03-07	nouveau: remove a hack that's not needed since the last interface change.	Ben Skeggs
2007-02-28	nouveau: intrusive drm interface changes	Ben Skeggs
	graphics objects: - No longer takes flags/dmaobj parameters, requires some major changes to the ddx to setup the object through the FIFO. This change is likely to cause breakages on some cards (tested on NV05,NV28,NV35, NV40 and NV4E). dma objects: - now takes a "class" parameter, not really used yet but we may need it at some point. - parameters are checked, so clients can't randomly create DMA objects pointing at whatever they feel like. misc: - Added FB_SIZE/AGP_SIZE getparams - Read PFIFO_INTR in PFIFO irq handler, not PMC_INTR - Dump PGRAPH trap info on PGRAPH_INTR_NOTIFY if NSOURCE isn't NOTIFICATION_PENDING.
2007-02-06	nouveau: more work on the nv04 context switch code.	Stephane Marchesin
2007-02-03	nouveau: cleanup the nv04 pgraph save/restore mechanism.	Stephane Marchesin
2007-02-03	nouveau: rename registers to their proper names.	Stephane Marchesin
2007-02-03	nouveau: add NV04 registers required for PGRAPH context switching.	Stephane Marchesin
2007-01-26	nouveau: add extra pgraph registers	Patrice Mandin
2007-01-13	nouveau: nv20 graph ctx switch.	Matthieu Castet
	Untested...
2007-01-13	nouveau: add and indent pgraph regs	Matthieu Castet
2007-01-13	nouveau: Oops, fix the nv04 RAMFC_DMA_FETCH value.	Stephane Marchesin
2007-01-12	nouveau: get nv30 context switching to work.	Jeremy Kolb
	* Pulled in some registers from nv10reg.h. Needed for context switching. * Filled in nv30 graphics context (based on nv40_graph.c). * Figure out nv30 context table, set up on context creation. Allows the cards automatic switching to work.
2007-01-06	nouveau: Use PMC_BOOT_0 to determine which ctx_voodoo to load.	Ben Skeggs
2007-01-05	Cleanup the nv04 fifo code a bit.	Stephane Marchesin
2006-11-28	For nv10, bit 16 of RAMFC need to be set for 64 bytes fifo context.	Matthieu Castet
	When cleaning a fifo, we shouldn't assume everybody use nv40 ;) Fill DMA_SUBROUTINE fill correct value.
2006-11-14	Completely untested NV10/20/30 FIFO context switching changes.	Ben Skeggs
2006-11-14	Restructure initialisation a bit.	Ben Skeggs
	- Do important card init in firstopen - Give each channel it's own cmdbuf dma object - Move RAMHT config state to the same place as RAMRO/RAMFC - Make sure instance mem for objects is *after* RAM{FC,HT,RO}
2006-11-06	fixup fifo size so it is page aligned	Dave Airlie
2006-10-18	Remove hack which delays activation of a additional channel. The previously ↵	Ben Skeggs
	active channel's state is saved to RAMFC before PFIFO gets clobbered.
2006-10-17	Useful output on a FIFO error interrupt.	Ben Skeggs
2006-10-17	Setup NV40 RAMFC (in wrong location.. but anyway), rearrange the RAMFC setup ↵	Ben Skeggs
	code a bit.
2006-10-17	Some info on NV40's RAMFC	Ben Skeggs
2006-10-12	Still more work on the context switching code.	Stephane Marchesin
2006-10-11	Context switching work.	Stephane Marchesin
	Added preliminary support for context switches (triggers the interrupts, but hangs after the switch ; something's not quite right yet). Removed the PFIFO_REINIT ioctl. I hope it's that a good idea... Requires the upcoming commit to the DDX.

• [next]

generated by cgit v1.2.3 (git 2.25.1) at 2025-12-16 14:53:06 +0000

/* r128_state.c -- State support for r128 -*- linux-c -*-
 * Created: Thu Jan 27 02:53:43 2000 by gareth@valinux.com
 *
 * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California.
 * All Rights Reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the next
 * paragraph) shall be included in all copies or substantial portions of the
 * Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * PRECISION INSIGHT AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
 * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
 * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 *
 * Authors:
 *    Gareth Hughes <gareth@valinux.com>
 */

#include "r128.h"
#include "drmP.h"
#include "drm.h"
#include "r128_drm.h"
#include "r128_drv.h"


/* ================================================================
 * CCE hardware state programming functions
 */

static void r128_emit_clip_rects( drm_r128_private_t *dev_priv,
				  drm_clip_rect_t *boxes, int count )
{
	u32 aux_sc_cntl = 0x00000000;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( (count < 3? count: 3) * 5 + 2 );

	if ( count >= 1 ) {
		OUT_RING( CCE_PACKET0( R128_AUX1_SC_LEFT, 3 ) );
		OUT_RING( boxes[0].x1 );
		OUT_RING( boxes[0].x2 - 1 );
		OUT_RING( boxes[0].y1 );
		OUT_RING( boxes[0].y2 - 1 );

		aux_sc_cntl |= (R128_AUX1_SC_EN | R128_AUX1_SC_MODE_OR);
	}
	if ( count >= 2 ) {
		OUT_RING( CCE_PACKET0( R128_AUX2_SC_LEFT, 3 ) );
		OUT_RING( boxes[1].x1 );
		OUT_RING( boxes[1].x2 - 1 );
		OUT_RING( boxes[1].y1 );
		OUT_RING( boxes[1].y2 - 1 );

		aux_sc_cntl |= (R128_AUX2_SC_EN | R128_AUX2_SC_MODE_OR);
	}
	if ( count >= 3 ) {
		OUT_RING( CCE_PACKET0( R128_AUX3_SC_LEFT, 3 ) );
		OUT_RING( boxes[2].x1 );
		OUT_RING( boxes[2].x2 - 1 );
		OUT_RING( boxes[2].y1 );
		OUT_RING( boxes[2].y2 - 1 );

		aux_sc_cntl |= (R128_AUX3_SC_EN | R128_AUX3_SC_MODE_OR);
	}

	OUT_RING( CCE_PACKET0( R128_AUX_SC_CNTL, 0 ) );
	OUT_RING( aux_sc_cntl );

	ADVANCE_RING();
}

static __inline__ void r128_emit_core( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_context_regs_t *ctx = &sarea_priv->context_state;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 2 );

	OUT_RING( CCE_PACKET0( R128_SCALE_3D_CNTL, 0 ) );
	OUT_RING( ctx->scale_3d_cntl );

	ADVANCE_RING();
}

static __inline__ void r128_emit_context( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_context_regs_t *ctx = &sarea_priv->context_state;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 13 );

	OUT_RING( CCE_PACKET0( R128_DST_PITCH_OFFSET_C, 11 ) );
	OUT_RING( ctx->dst_pitch_offset_c );
	OUT_RING( ctx->dp_gui_master_cntl_c );
	OUT_RING( ctx->sc_top_left_c );
	OUT_RING( ctx->sc_bottom_right_c );
	OUT_RING( ctx->z_offset_c );
	OUT_RING( ctx->z_pitch_c );
	OUT_RING( ctx->z_sten_cntl_c );
	OUT_RING( ctx->tex_cntl_c );
	OUT_RING( ctx->misc_3d_state_cntl_reg );
	OUT_RING( ctx->texture_clr_cmp_clr_c );
	OUT_RING( ctx->texture_clr_cmp_msk_c );
	OUT_RING( ctx->fog_color_c );

	ADVANCE_RING();
}

static __inline__ void r128_emit_setup( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_context_regs_t *ctx = &sarea_priv->context_state;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 3 );

	OUT_RING( CCE_PACKET1( R128_SETUP_CNTL, R128_PM4_VC_FPU_SETUP ) );
	OUT_RING( ctx->setup_cntl );
	OUT_RING( ctx->pm4_vc_fpu_setup );

	ADVANCE_RING();
}

static __inline__ void r128_emit_masks( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_context_regs_t *ctx = &sarea_priv->context_state;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 5 );

	OUT_RING( CCE_PACKET0( R128_DP_WRITE_MASK, 0 ) );
	OUT_RING( ctx->dp_write_mask );

	OUT_RING( CCE_PACKET0( R128_STEN_REF_MASK_C, 1 ) );
	OUT_RING( ctx->sten_ref_mask_c );
	OUT_RING( ctx->plane_3d_mask_c );

	ADVANCE_RING();
}

static __inline__ void r128_emit_window( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_context_regs_t *ctx = &sarea_priv->context_state;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 2 );

	OUT_RING( CCE_PACKET0( R128_WINDOW_XY_OFFSET, 0 ) );
	OUT_RING( ctx->window_xy_offset );

	ADVANCE_RING();
}

static __inline__ void r128_emit_tex0( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_context_regs_t *ctx = &sarea_priv->context_state;
	drm_r128_texture_regs_t *tex = &sarea_priv->tex_state[0];
	int i;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 7 + R128_MAX_TEXTURE_LEVELS );

	OUT_RING( CCE_PACKET0( R128_PRIM_TEX_CNTL_C,
			       2 + R128_MAX_TEXTURE_LEVELS ) );
	OUT_RING( tex->tex_cntl );
	OUT_RING( tex->tex_combine_cntl );
	OUT_RING( ctx->tex_size_pitch_c );
	for ( i = 0 ; i < R128_MAX_TEXTURE_LEVELS ; i++ ) {
		OUT_RING( tex->tex_offset[i] );
	}

	OUT_RING( CCE_PACKET0( R128_CONSTANT_COLOR_C, 1 ) );
	OUT_RING( ctx->constant_color_c );
	OUT_RING( tex->tex_border_color );

	ADVANCE_RING();
}

static __inline__ void r128_emit_tex1( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	drm_r128_texture_regs_t *tex = &sarea_priv->tex_state[1];
	int i;
	RING_LOCALS;
	DRM_DEBUG( "    %s\n", __FUNCTION__ );

	BEGIN_RING( 5 + R128_MAX_TEXTURE_LEVELS );

	OUT_RING( CCE_PACKET0( R128_SEC_TEX_CNTL_C,
			       1 + R128_MAX_TEXTURE_LEVELS ) );
	OUT_RING( tex->tex_cntl );
	OUT_RING( tex->tex_combine_cntl );
	for ( i = 0 ; i < R128_MAX_TEXTURE_LEVELS ; i++ ) {
		OUT_RING( tex->tex_offset[i] );
	}

	OUT_RING( CCE_PACKET0( R128_SEC_TEXTURE_BORDER_COLOR_C, 0 ) );
	OUT_RING( tex->tex_border_color );

	ADVANCE_RING();
}

static __inline__ void r128_emit_state( drm_r128_private_t *dev_priv )
{
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	unsigned int dirty = sarea_priv->dirty;

	DRM_DEBUG( "%s: dirty=0x%08x\n", __FUNCTION__, dirty );

	if ( dirty & R128_UPLOAD_CORE ) {
		r128_emit_core( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_CORE;
	}

	if ( dirty & R128_UPLOAD_CONTEXT ) {
		r128_emit_context( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_CONTEXT;
	}

	if ( dirty & R128_UPLOAD_SETUP ) {
		r128_emit_setup( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_SETUP;
	}

	if ( dirty & R128_UPLOAD_MASKS ) {
		r128_emit_masks( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_MASKS;
	}

	if ( dirty & R128_UPLOAD_WINDOW ) {
		r128_emit_window( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_WINDOW;
	}

	if ( dirty & R128_UPLOAD_TEX0 ) {
		r128_emit_tex0( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_TEX0;
	}

	if ( dirty & R128_UPLOAD_TEX1 ) {
		r128_emit_tex1( dev_priv );
		sarea_priv->dirty &= ~R128_UPLOAD_TEX1;
	}

	/* Turn off the texture cache flushing */
	sarea_priv->context_state.tex_cntl_c &= ~R128_TEX_CACHE_FLUSH;

	sarea_priv->dirty &= ~R128_REQUIRE_QUIESCENCE;
}


#if R128_PERFORMANCE_BOXES
/* ================================================================
 * Performance monitoring functions
 */

static void r128_clear_box( drm_r128_private_t *dev_priv,
			    int x, int y, int w, int h,
			    int r, int g, int b )
{
	u32 pitch, offset;
	u32 fb_bpp, color;
	RING_LOCALS;

	switch ( dev_priv->fb_bpp ) {
	case 16:
		fb_bpp = R128_GMC_DST_16BPP;
		color = (((r & 0xf8) << 8) |
			 ((g & 0xfc) << 3) |
			 ((b & 0xf8) >> 3));
		break;
	case 24:
		fb_bpp = R128_GMC_DST_24BPP;
		color = ((r << 16) | (g << 8) | b);
		break;
	case 32:
		fb_bpp = R128_GMC_DST_32BPP;
		color = (((0xff) << 24) | (r << 16) | (g <<  8) | b);
		break;
	default:
		return;
	}

	offset = dev_priv->back_offset;
	pitch = dev_priv->back_pitch >> 3;

	BEGIN_RING( 6 );

	OUT_RING( CCE_PACKET3( R128_CNTL_PAINT_MULTI, 4 ) );
	OUT_RING( R128_GMC_DST_PITCH_OFFSET_CNTL |
		  R128_GMC_BRUSH_SOLID_COLOR |
		  fb_bpp |
		  R128_GMC_SRC_DATATYPE_COLOR |
		  R128_ROP3_P |
		  R128_GMC_CLR_CMP_CNTL_DIS |
		  R128_GMC_AUX_CLIP_DIS );

	OUT_RING( (pitch << 21) | (offset >> 5) );
	OUT_RING( color );

	OUT_RING( (x << 16) | y );
	OUT_RING( (w << 16) | h );

	ADVANCE_RING();
}

static void r128_cce_performance_boxes( drm_r128_private_t *dev_priv )
{
	if ( atomic_read( &dev_priv->idle_count ) == 0 ) {
		r128_clear_box( dev_priv, 64, 4, 8, 8, 0, 255, 0 );
	} else {
		atomic_set( &dev_priv->idle_count, 0 );
	}
}

#endif


/* ================================================================
 * CCE command dispatch functions
 */

static void r128_print_dirty( const char *msg, unsigned int flags )
{
	DRM_INFO( "%s: (0x%x) %s%s%s%s%s%s%s%s%s\n",
		  msg,
		  flags,
		  (flags & R128_UPLOAD_CORE)        ? "core, " : "",
		  (flags & R128_UPLOAD_CONTEXT)     ? "context, " : "",
		  (flags & R128_UPLOAD_SETUP)       ? "setup, " : "",
		  (flags & R128_UPLOAD_TEX0)        ? "tex0, " : "",
		  (flags & R128_UPLOAD_TEX1)        ? "tex1, " : "",
		  (flags & R128_UPLOAD_MASKS)       ? "masks, " : "",
		  (flags & R128_UPLOAD_WINDOW)      ? "window, " : "",
		  (flags & R128_UPLOAD_CLIPRECTS)   ? "cliprects, " : "",
		  (flags & R128_REQUIRE_QUIESCENCE) ? "quiescence, " : "" );
}

static void r128_cce_dispatch_clear( drm_device_t *dev,
				     drm_r128_clear_t *clear )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	int nbox = sarea_priv->nbox;
	drm_clip_rect_t *pbox = sarea_priv->boxes;
	unsigned int flags = clear->flags;
	int i;
	RING_LOCALS;
	DRM_DEBUG( "%s\n", __FUNCTION__ );

	if ( dev_priv->page_flipping && dev_priv->current_page == 1 ) {
		unsigned int tmp = flags;

		flags &= ~(R128_FRONT | R128_BACK);
		if ( tmp & R128_FRONT ) flags |= R128_BACK;
		if ( tmp & R128_BACK )  flags |= R128_FRONT;
	}

	for ( i = 0 ; i < nbox ; i++ ) {
		int x = pbox[i].x1;
		int y = pbox[i].y1;
		int w = pbox[i].x2 - x;
		int h = pbox[i].y2 - y;

		DRM_DEBUG( "dispatch clear %d,%d-%d,%d flags 0x%x\n",
			   pbox[i].x1, pbox[i].y1, pbox[i].x2,
			   pbox[i].y2, flags );

		if ( flags & (R128_FRONT | R128_BACK) ) {
			BEGIN_RING( 2 );

			OUT_RING( CCE_PACKET0( R128_DP_WRITE_MASK, 0 ) );
			OUT_RING( clear->color_mask );

			ADVANCE_RING();
		}

		if ( flags & R128_FRONT ) {
			BEGIN_RING( 6 );

			OUT_RING( CCE_PACKET3( R128_CNTL_PAINT_MULTI, 4 ) );
			OUT_RING( R128_GMC_DST_PITCH_OFFSET_CNTL |
				  R128_GMC_BRUSH_SOLID_COLOR |
				  (dev_priv->color_fmt << 8) |
				  R128_GMC_SRC_DATATYPE_COLOR |
				  R128_ROP3_P |
				  R128_GMC_CLR_CMP_CNTL_DIS |
				  R128_GMC_AUX_CLIP_DIS );

			OUT_RING( dev_priv->front_pitch_offset_c );
			OUT_RING( clear->clear_color );

			OUT_RING( (x << 16) | y );
			OUT_RING( (w << 16) | h );

			ADVANCE_RING();
		}

		if ( flags & R128_BACK ) {
			BEGIN_RING( 6 );

			OUT_RING( CCE_PACKET3( R128_CNTL_PAINT_MULTI, 4 ) );
			OUT_RING( R128_GMC_DST_PITCH_OFFSET_CNTL |
				  R128_GMC_BRUSH_SOLID_COLOR |
				  (dev_priv->color_fmt << 8) |
				  R128_GMC_SRC_DATATYPE_COLOR |
				  R128_ROP3_P |
				  R128_GMC_CLR_CMP_CNTL_DIS |
				  R128_GMC_AUX_CLIP_DIS );

			OUT_RING( dev_priv->back_pitch_offset_c );
			OUT_RING( clear->clear_color );

			OUT_RING( (x << 16) | y );
			OUT_RING( (w << 16) | h );

			ADVANCE_RING();
		}

		if ( flags & R128_DEPTH ) {
			BEGIN_RING( 6 );

			OUT_RING( CCE_PACKET3( R128_CNTL_PAINT_MULTI, 4 ) );
			OUT_RING( R128_GMC_DST_PITCH_OFFSET_CNTL |
				  R128_GMC_BRUSH_SOLID_COLOR |
				  (dev_priv->depth_fmt << 8) |
				  R128_GMC_SRC_DATATYPE_COLOR |
				  R128_ROP3_P |
				  R128_GMC_CLR_CMP_CNTL_DIS |
				  R128_GMC_AUX_CLIP_DIS |
				  R128_GMC_WR_MSK_DIS );

			OUT_RING( dev_priv->depth_pitch_offset_c );
			OUT_RING( clear->clear_depth );

			OUT_RING( (x << 16) | y );
			OUT_RING( (w << 16) | h );

			ADVANCE_RING();
		}
	}
}

static void r128_cce_dispatch_swap( drm_device_t *dev )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	int nbox = sarea_priv->nbox;
	drm_clip_rect_t *pbox = sarea_priv->boxes;
	int i;
	RING_LOCALS;
	DRM_DEBUG( "%s\n", __FUNCTION__ );

#if R128_PERFORMANCE_BOXES
	/* Do some trivial performance monitoring...
	 */
	r128_cce_performance_boxes( dev_priv );
#endif

	for ( i = 0 ; i < nbox ; i++ ) {
		int x = pbox[i].x1;
		int y = pbox[i].y1;
		int w = pbox[i].x2 - x;
		int h = pbox[i].y2 - y;

		BEGIN_RING( 7 );

		OUT_RING( CCE_PACKET3( R128_CNTL_BITBLT_MULTI, 5 ) );
		OUT_RING( R128_GMC_SRC_PITCH_OFFSET_CNTL |
			  R128_GMC_DST_PITCH_OFFSET_CNTL |
			  R128_GMC_BRUSH_NONE |
			  (dev_priv->color_fmt << 8) |
			  R128_GMC_SRC_DATATYPE_COLOR |
			  R128_ROP3_S |
			  R128_DP_SRC_SOURCE_MEMORY |
			  R128_GMC_CLR_CMP_CNTL_DIS |
			  R128_GMC_AUX_CLIP_DIS |
			  R128_GMC_WR_MSK_DIS );

		/* Make this work even if front & back are flipped:
		 */
		if (dev_priv->current_page == 0) {
			OUT_RING( dev_priv->back_pitch_offset_c );
			OUT_RING( dev_priv->front_pitch_offset_c );
		} 
		else {
			OUT_RING( dev_priv->front_pitch_offset_c );
			OUT_RING( dev_priv->back_pitch_offset_c );
		}

		OUT_RING( (x << 16) | y );
		OUT_RING( (x << 16) | y );
		OUT_RING( (w << 16) | h );

		ADVANCE_RING();
	}

	/* Increment the frame counter.  The client-side 3D driver must
	 * throttle the framerate by waiting for this value before
	 * performing the swapbuffer ioctl.
	 */
	dev_priv->sarea_priv->last_frame++;

	BEGIN_RING( 2 );

	OUT_RING( CCE_PACKET0( R128_LAST_FRAME_REG, 0 ) );
	OUT_RING( dev_priv->sarea_priv->last_frame );

	ADVANCE_RING();
}

static void r128_cce_dispatch_flip( drm_device_t *dev )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	RING_LOCALS;
	DRM_DEBUG("%s: page=%d pfCurrentPage=%d\n", 
		__FUNCTION__,
		dev_priv->current_page,
		dev_priv->sarea_priv->pfCurrentPage);

#if R128_PERFORMANCE_BOXES
	/* Do some trivial performance monitoring...
	 */
	r128_cce_performance_boxes( dev_priv );
#endif

	BEGIN_RING( 4 );

	R128_WAIT_UNTIL_PAGE_FLIPPED();
	OUT_RING( CCE_PACKET0( R128_CRTC_OFFSET, 0 ) );

	if ( dev_priv->current_page == 0 ) {
		OUT_RING( dev_priv->back_offset );
	} else {
		OUT_RING( dev_priv->front_offset );
	}

	ADVANCE_RING();

	/* Increment the frame counter.  The client-side 3D driver must
	 * throttle the framerate by waiting for this value before
	 * performing the swapbuffer ioctl.
	 */
	dev_priv->sarea_priv->last_frame++;
	dev_priv->sarea_priv->pfCurrentPage = dev_priv->current_page =
					      1 - dev_priv->current_page;

	BEGIN_RING( 2 );

	OUT_RING( CCE_PACKET0( R128_LAST_FRAME_REG, 0 ) );
	OUT_RING( dev_priv->sarea_priv->last_frame );

	ADVANCE_RING();
}

static void r128_cce_dispatch_vertex( drm_device_t *dev,
				      drm_buf_t *buf )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	drm_r128_buf_priv_t *buf_priv = buf->dev_private;
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	int format = sarea_priv->vc_format;
	int offset = buf->bus_address;
	int size = buf->used;
	int prim = buf_priv->prim;
	int i = 0;
	RING_LOCALS;
	DRM_DEBUG( "buf=%d nbox=%d\n", buf->idx, sarea_priv->nbox );

	if ( 0 )
		r128_print_dirty( "dispatch_vertex", sarea_priv->dirty );

	if ( buf->used ) {
		buf_priv->dispatched = 1;

		if ( sarea_priv->dirty & ~R128_UPLOAD_CLIPRECTS ) {
			r128_emit_state( dev_priv );
		}

		do {
			/* Emit the next set of up to three cliprects */
			if ( i < sarea_priv->nbox ) {
				r128_emit_clip_rects( dev_priv,
						      &sarea_priv->boxes[i],
						      sarea_priv->nbox - i );
			}

			/* Emit the vertex buffer rendering commands */
			BEGIN_RING( 5 );

			OUT_RING( CCE_PACKET3( R128_3D_RNDR_GEN_INDX_PRIM, 3 ) );
			OUT_RING( offset );
			OUT_RING( size );
			OUT_RING( format );
			OUT_RING( prim | R128_CCE_VC_CNTL_PRIM_WALK_LIST |
				  (size << R128_CCE_VC_CNTL_NUM_SHIFT) );

			ADVANCE_RING();

			i += 3;
		} while ( i < sarea_priv->nbox );
	}

	if ( buf_priv->discard ) {
		buf_priv->age = dev_priv->sarea_priv->last_dispatch;

		/* Emit the vertex buffer age */
		BEGIN_RING( 2 );

		OUT_RING( CCE_PACKET0( R128_LAST_DISPATCH_REG, 0 ) );
		OUT_RING( buf_priv->age );

		ADVANCE_RING();

		buf->pending = 1;
		buf->used = 0;
		/* FIXME: Check dispatched field */
		buf_priv->dispatched = 0;
	}

	dev_priv->sarea_priv->last_dispatch++;

	sarea_priv->dirty &= ~R128_UPLOAD_CLIPRECTS;
	sarea_priv->nbox = 0;
}

static void r128_cce_dispatch_indirect( drm_device_t *dev,
					drm_buf_t *buf,
					int start, int end )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	drm_r128_buf_priv_t *buf_priv = buf->dev_private;
	RING_LOCALS;
	DRM_DEBUG( "indirect: buf=%d s=0x%x e=0x%x\n",
		   buf->idx, start, end );

	if ( start != end ) {
		int offset = buf->bus_address + start;
		int dwords = (end - start + 3) / sizeof(u32);

		/* Indirect buffer data must be an even number of
		 * dwords, so if we've been given an odd number we must
		 * pad the data with a Type-2 CCE packet.
		 */
		if ( dwords & 1 ) {
			u32 *data = (u32 *)
				((char *)dev->agp_buffer_map->handle
				 + buf->offset + start);
			data[dwords++] = cpu_to_le32( R128_CCE_PACKET2 );
		}

		buf_priv->dispatched = 1;

		/* Fire off the indirect buffer */
		BEGIN_RING( 3 );

		OUT_RING( CCE_PACKET0( R128_PM4_IW_INDOFF, 1 ) );
		OUT_RING( offset );
		OUT_RING( dwords );

		ADVANCE_RING();
	}

	if ( buf_priv->discard ) {
		buf_priv->age = dev_priv->sarea_priv->last_dispatch;

		/* Emit the indirect buffer age */
		BEGIN_RING( 2 );

		OUT_RING( CCE_PACKET0( R128_LAST_DISPATCH_REG, 0 ) );
		OUT_RING( buf_priv->age );

		ADVANCE_RING();

		buf->pending = 1;
		buf->used = 0;
		/* FIXME: Check dispatched field */
		buf_priv->dispatched = 0;
	}

	dev_priv->sarea_priv->last_dispatch++;
}

static void r128_cce_dispatch_indices( drm_device_t *dev,
				       drm_buf_t *buf,
				       int start, int end,
				       int count )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	drm_r128_buf_priv_t *buf_priv = buf->dev_private;
	drm_r128_sarea_t *sarea_priv = dev_priv->sarea_priv;
	int format = sarea_priv->vc_format;
	int offset = dev->agp_buffer_map->offset - dev_priv->cce_buffers_offset;
	int prim = buf_priv->prim;
	u32 *data;
	int dwords;
	int i = 0;
	RING_LOCALS;
	DRM_DEBUG( "indices: s=%d e=%d c=%d\n", start, end, count );

	if ( 0 )
		r128_print_dirty( "dispatch_indices", sarea_priv->dirty );

	if ( start != end ) {
		buf_priv->dispatched = 1;

		if ( sarea_priv->dirty & ~R128_UPLOAD_CLIPRECTS ) {
			r128_emit_state( dev_priv );
		}

		dwords = (end - start + 3) / sizeof(u32);

		data = (u32 *)((char *)dev->agp_buffer_map->handle
			       + buf->offset + start);

		data[0] = cpu_to_le32( CCE_PACKET3( R128_3D_RNDR_GEN_INDX_PRIM,
						    dwords-2 ) );

		data[1] = cpu_to_le32( offset );
		data[2] = cpu_to_le32( R128_MAX_VB_VERTS );
		data[3] = cpu_to_le32( format );
		data[4] = cpu_to_le32( (prim | R128_CCE_VC_CNTL_PRIM_WALK_IND |
					(count << 16)) );

		if ( count & 0x1 ) {
#ifdef __LITTLE_ENDIAN
			data[dwords-1] &= 0x0000ffff;
#else
			data[dwords-1] &= 0xffff0000;
#endif
		}

		do {
			/* Emit the next set of up to three cliprects */
			if ( i < sarea_priv->nbox ) {
				r128_emit_clip_rects( dev_priv,
						      &sarea_priv->boxes[i],
						      sarea_priv->nbox - i );
			}

			r128_cce_dispatch_indirect( dev, buf, start, end );

			i += 3;
		} while ( i < sarea_priv->nbox );
	}

	if ( buf_priv->discard ) {
		buf_priv->age = dev_priv->sarea_priv->last_dispatch;

		/* Emit the vertex buffer age */
		BEGIN_RING( 2 );

		OUT_RING( CCE_PACKET0( R128_LAST_DISPATCH_REG, 0 ) );
		OUT_RING( buf_priv->age );

		ADVANCE_RING();

		buf->pending = 1;
		/* FIXME: Check dispatched field */
		buf_priv->dispatched = 0;
	}

	dev_priv->sarea_priv->last_dispatch++;

	sarea_priv->dirty &= ~R128_UPLOAD_CLIPRECTS;
	sarea_priv->nbox = 0;
}

static int r128_cce_dispatch_blit( DRMFILE filp,
				   drm_device_t *dev,
				   drm_r128_blit_t *blit )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	drm_device_dma_t *dma = dev->dma;
	drm_buf_t *buf;
	drm_r128_buf_priv_t *buf_priv;
	u32 *data;
	int dword_shift, dwords;
	RING_LOCALS;
	DRM_DEBUG( "\n" );

	/* The compiler won't optimize away a division by a variable,
	 * even if the only legal values are powers of two.  Thus, we'll
	 * use a shift instead.
	 */
	switch ( blit->format ) {
	case R128_DATATYPE_ARGB8888:
		dword_shift = 0;
		break;
	case R128_DATATYPE_ARGB1555:
	case R128_DATATYPE_RGB565:
	case R128_DATATYPE_ARGB4444:
	case R128_DATATYPE_YVYU422:
	case R128_DATATYPE_VYUY422:
		dword_shift = 1;
		break;
	case R128_DATATYPE_CI8:
	case R128_DATATYPE_RGB8:
		dword_shift = 2;
		break;
	default:
		DRM_ERROR( "invalid blit format %d\n", blit->format );
		return DRM_ERR(EINVAL);
	}

	/* Flush the pixel cache, and mark the contents as Read Invalid.
	 * This ensures no pixel data gets mixed up with the texture
	 * data from the host data blit, otherwise part of the texture
	 * image may be corrupted.
	 */
	BEGIN_RING( 2 );

	OUT_RING( CCE_PACKET0( R128_PC_GUI_CTLSTAT, 0 ) );
	OUT_RING( R128_PC_RI_GUI | R128_PC_FLUSH_GUI );

	ADVANCE_RING();

	/* Dispatch the indirect buffer.
	 */
	buf = dma->buflist[blit->idx];
	buf_priv = buf->dev_private;

	if ( buf->filp != filp ) {
		DRM_ERROR( "process %d using buffer owned by %p\n",
			   DRM_CURRENTPID, buf->filp );
		return DRM_ERR(EINVAL);
	}
	if ( buf->pending ) {
		DRM_ERROR( "sending pending buffer %d\n", blit->idx );
		return DRM_ERR(EINVAL);
	}

	buf_priv->discard = 1;

	dwords = (blit->width * blit->height) >> dword_shift;

	data = (u32 *)((char *)dev->agp_buffer_map->handle + buf->offset);

	data[0] = cpu_to_le32( CCE_PACKET3( R128_CNTL_HOSTDATA_BLT, dwords + 6 ) );
	data[1] = cpu_to_le32( (R128_GMC_DST_PITCH_OFFSET_CNTL |
				R128_GMC_BRUSH_NONE |
				(blit->format << 8) |
				R128_GMC_SRC_DATATYPE_COLOR |
				R128_ROP3_S |
				R128_DP_SRC_SOURCE_HOST_DATA |
				R128_GMC_CLR_CMP_CNTL_DIS |
				R128_GMC_AUX_CLIP_DIS |
				R128_GMC_WR_MSK_DIS) );

	data[2] = cpu_to_le32( (blit->pitch << 21) | (blit->offset >> 5) );
	data[3] = cpu_to_le32( 0xffffffff );
	data[4] = cpu_to_le32( 0xffffffff );
	data[5] = cpu_to_le32( (blit->y << 16) | blit->x );
	data[6] = cpu_to_le32( (blit->height << 16) | blit->width );
	data[7] = cpu_to_le32( dwords );

	buf->used = (dwords + 8) * sizeof(u32);

	r128_cce_dispatch_indirect( dev, buf, 0, buf->used );

	/* Flush the pixel cache after the blit completes.  This ensures
	 * the texture data is written out to memory before rendering
	 * continues.
	 */
	BEGIN_RING( 2 );

	OUT_RING( CCE_PACKET0( R128_PC_GUI_CTLSTAT, 0 ) );
	OUT_RING( R128_PC_FLUSH_GUI );

	ADVANCE_RING();

	return 0;
}


/* ================================================================
 * Tiled depth buffer management
 *
 * FIXME: These should all set the destination write mask for when we
 * have hardware stencil support.
 */

static int r128_cce_dispatch_write_span( drm_device_t *dev,
					 drm_r128_depth_t *depth )
{
	drm_r128_private_t *dev_priv = dev->dev_private;
	int count, x, y;
	u32 *buffer;
	u8 *mask;
	int i, buffer_size, mask_size;
	RING_LOCALS;
	DRM_DEBUG( "\n" );

	count = depth->n;
	if (count > 4096 || count <= 0)
		return DRM_ERR(EMSGSIZE);