summaryrefslogtreecommitdiff
path: root/shared-core/via_drv.c
AgeCommit message (Collapse)Author
2008-01-30Simplify the fencing code and differentiate between flushes andThomas Hellstrom
waiting types. Add a "command_stream_barrier" method to the bo driver.
2008-01-22Merge branch 'master' into vblank-rework, including mach64 supportJesse Barnes
Conflicts: linux-core/drmP.h linux-core/drm_drv.c shared-core/i915_drv.h shared-core/i915_irq.c shared-core/mga_irq.c shared-core/radeon_irq.c shared-core/via_irq.c Mostly trivial conflicts. mach64 support from Mathieu Bérard.
2007-12-21Rename inappropriately named 'mask' fields to 'proposed_flags' instead.Keith Packard
Flags pending validation were stored in a misleadingly named field, 'mask'. As 'mask' is already used to indicate pieces of a flags field which are changing, it seems better to use a name reflecting the actual purpose of this field. I chose 'proposed_flags' as they may not actually end up in 'flags', and in an case will be modified when they are moved over. This affects the API, but not ABI of the user-mode interface.
2007-10-30Merge branch 'master' into vblank-rework, fixup remaining driversJesse Barnes
Conflicts: linux-core/drmP.h linux-core/drm_drv.c linux-core/drm_irq.c shared-core/i915_drv.h shared-core/i915_irq.c shared-core/mga_drv.h shared-core/mga_irq.c shared-core/radeon_drv.h shared-core/radeon_irq.c Merge in the latest master bits and update the remaining drivers (except mach64 which math_b is working on). Also remove the 9xx hack from the i915 driver; it seems to be correct.
2007-07-16drm: detypedef ttm/bo/fence codeDave Airlie
2007-02-16Simple fence object sample driver for via, based on idling the GPU.Thomas Hellstrom
Buffer object driver for via. Some changes to buffer object driver callbacks. Improve fence flushing.
2007-02-13Bugzilla Bug #9457Thomas Hellstrom
Add refcounting of user waiters to the DRM hardware lock, so that we can use the DRM_LOCK_CONT flag more conservatively. Also add a kernel waiter refcount that if nonzero transfers the lock for the kernel context, when it is released. This is useful when waiting for idle and can be used for very simple fence object driver implementations for the new memory manager. It also resolves the AIGLX startup deadlock for the sis and the via drivers. i810, i830 still require that the hardware lock is really taken so the deadlock remains for those two. I'm not sure about ffb. Anyone familiar with that code?
2006-10-14remove config.h from build no longer exists kbuild does itDave Airlie
2006-06-15via:Thomas Hellstrom
-Remove out of memory error message. -Move sman cleanup from final_context to lastclose. -Add the P4VM800PRO (?) PCI ID.
2006-06-06Merge in the drm-sman-branchThomas Hellstrom
2006-02-17via: Change via_drm.h versioning scheme after lenghty discussion onThomas Hellstrom
unichrome-users.
2005-11-11cleanup ioctl/max_ioctl to use header file for extern symbolsDave Airlie
2005-08-15Port the VIA DRM to FreeBSD. Original patch by Jake, with some cleanup byEric Anholt
me to match other drivers and avoid ifdeffing. The linux via_drv.c will be moved from shared-core to linux-core soon by repocopy. Submitted by: Jake Burkholder <jake@FreeBSD.org> Tested by: unichrome
2005-08-14VIA bugvixes by Joris van Rantwijk Initial commit.Thomas Hellstrom
2005-08-05Rename the driver hooks in the DRM to something a little moreEric Anholt
understandable: preinit -> load postinit -> (removed) presetup -> firstopen postsetup -> (removed) open_helper -> open prerelease -> preclose free_filp_priv -> postclose pretakedown -> lastclose postcleanup -> unload release -> reclaim_buffers_locked version -> (removed) postinit and version were replaced with generic code in the Linux DRM (drivers now set their version numbers and description in the driver structure, like on BSD). postsetup wasn't used at all. Fixes the savage hooks for initializing and tearing down mappings at the right times. Testing involved at least starting X, running glxgears, killing glxgears, exiting X, and repeating. Tested on: FreeBSD (g200, g400, r200, r128) Linux (r200, savage4)
2005-08-04Split the control of master vs root priv. Everything is still marked asJon Smirl
needing root.
2005-07-03Add sysfs attribute dri_library_name on Linux. code in share-core/via_drv.cJon Smirl
is ok to be shared, it will be passive on BSD.
2005-03-28Via updates:Thomas Hellstrom
New PCI command parser. Moved from via_dma.c to via_verifier.c so functions with similar functionality are close to eachother. Moved video related functions to via_video.c, which might be extended in the future, as new video functionality is added. New device-specific generic IRQ IOCTL, similar to the general VBLANK IOCTL, but with support for multiple device IRQ sources and functionality. Support for Unichrome Pro PM800/CN400 video DMA commands in verifier and PCI parser. Support for Unichrome Pro PM800/CN400 HQV IRQs in the new generic IRQ IOCTL. Bumped minor. New version 2.6.0.
2005-02-03Via driver: Add missing drm_poll function to via driver in core.Thomas Hellstrom
2005-01-05VIA update:Thomas Hellstrom
Release video futexes when context is destroyed (This was previously done by the X server). Added New Unichrome Pro VIDEO DMA commands to the verifier. Added Quiescent heavyweight lock mode.
2004-12-13VIA drm updates:Thomas Hellstrom
1. Improved security check of AGP texture adresses. 2. Hopefully last fix of ring-buffer jump oddities. 3. Added ioctl to check available space and command regulator lag in ring-buffer. This is needed for 3D application responsiveness.
2004-12-03Added 3D functionality to the via command verifier. Modified the viaThomas Hellstrom
ring-buffer code somewhat to workaround hardware problems. Bumped via minor version number.
2004-10-12Breakout heads into their own data structures.Jon Smirl
2004-10-10Rename fn_tbl to driver. Core driver now uses pci_driver name whichJon Smirl
reflects the personality name.
2004-10-09cleanup VIA driver to look a bit like others before kernel mergeDave Airlie
2004-10-08Changed unsigned to uint32_t in some ioctl parameters. Introduced firstThomas Hellstrom
rudimentary command verifier for dma buffers. Changed the decoder futex ioctl parameters. Bumped the via major version number.
2004-09-30Lindent of core build. Drivers checked for no binary diffs. A few filesJon Smirl
weren't Lindent's because their comments didn't convert very well. A bunch of other minor clean up with no code implact included.
2004-09-30Make fops per driver instead of global, remove default flush, poll, readJon Smirl
functions
2004-09-27First check in for DRM that splits core from personality modulesJon Smirl
2004-09-231) switches from class_sysfs to drm sysfs implementation to allowJon Smirl
customization 2) compiles again on 2.4, but doesn't work
2004-09-20remove HAVE_COUNTERSDave Airlie
2004-09-07Added IOCTL for writing 2D DMA command buffers over PCI. Bumped minorThomas Hellstrom
version number.
2004-08-24Merged drmfntbl-0-0-2Dave Airlie
2004-08-24This patch adds three new ioctl's to the VIA Unichrome/Pro DRM driver:Erdi Chen
DRM_IOCTL_VIA_DMA_INIT DRM_IOCTL_VIA_CMDBUFFER DRM_IOCTL_VIA_FLUSH The first ioctl sets up an area in AGP memory that will be used as the ring buffer. The second ioctl copies a command buffer from user space memory to the ring buffer. The third ioctl waits for engine idle until it returns. The motivation for this patch is to avoid the wait for engine idle call before each buffer flush in the current DRI driver. With this patch, the DRI driver can continue to flush its buffer as long as there is free space in the ring buffer. This patch adds an additional copy operation on the command buffer. This buffer copying is necessary to support multiple DRI clients rendering simultaneously. Otherwise, more CPU time will be spent in the busy loop waiting for engine idle between DRI context switch. Even in the single client case, the tradeoff is reasonable in comparision to the kernel call to check for free buffer space for the client to render directly to the ring buffer.
2004-08-17Merged drmfntbl-0-0-1Dave Airlie
2004-04-121. Added a PCI ID.Thomas Hellstrom
2. Big change to the XvMC part of the SAREA. OpenGL clients will not suffer from this, and via XvMC is still alpha. Needed to make future additions to XvMC (More decoders and overlays) possible. 3. Bumped version number to 1.3.0.
2004-03-23Merged via-1-2-0Thomas Hellstrom
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-09 04:13:32 +0000
603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 
/* savage_state.c -- State and drawing support for Savage
 *
 * Copyright 2004  Felix Kuehling
 * All Rights Reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sub license,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the
 * next paragraph) shall be included in all copies or substantial portions
 * of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 * NON-INFRINGEMENT. IN NO EVENT SHALL FELIX KUEHLING BE LIABLE FOR
 * ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
 * CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 */
#include "drmP.h"
#include "savage_drm.h"
#include "savage_drv.h"

void savage_emit_clip_rect_s3d(drm_savage_private_t *dev_priv,
			       const drm_clip_rect_t *pbox)
{
	uint32_t scstart = dev_priv->state.s3d.new_scstart;
	uint32_t scend   = dev_priv->state.s3d.new_scend;
	scstart = (scstart & ~SAVAGE_SCISSOR_MASK_S3D) |
		((uint32_t)pbox->x1 & 0x000007ff) | 
		(((uint32_t)pbox->y1 << 16) & 0x07ff0000);
	scend   = (scend   & ~SAVAGE_SCISSOR_MASK_S3D) |
		(((uint32_t)pbox->x2-1) & 0x000007ff) |
		((((uint32_t)pbox->y2-1) << 16) & 0x07ff0000);
	if (scstart != dev_priv->state.s3d.scstart ||
	    scend   != dev_priv->state.s3d.scend) {
		DMA_LOCALS;
		BEGIN_DMA(4);
		DMA_WRITE(BCI_CMD_WAIT|BCI_CMD_WAIT_3D);
		DMA_SET_REGISTERS(SAVAGE_SCSTART_S3D, 2);
		DMA_WRITE(scstart);
		DMA_WRITE(scend);
		dev_priv->state.s3d.scstart = scstart;
		dev_priv->state.s3d.scend   = scend;
		dev_priv->waiting = 1;
		DMA_COMMIT();
	}
}

void savage_emit_clip_rect_s4(drm_savage_private_t *dev_priv,
			      const drm_clip_rect_t *pbox)
{
	uint32_t drawctrl0 = dev_priv->state.s4.new_drawctrl0;
	uint32_t drawctrl1 = dev_priv->state.s4.new_drawctrl1;
	drawctrl0 = (drawctrl0 & ~SAVAGE_SCISSOR_MASK_S4) |
		((uint32_t)pbox->x1 & 0x000007ff) |
		(((uint32_t)pbox->y1 << 12) & 0x00fff000);
	drawctrl1 = (drawctrl1 & ~SAVAGE_SCISSOR_MASK_S4) |
		(((uint32_t)pbox->x2-1) & 0x000007ff) |
		((((uint32_t)pbox->y2-1) << 12) & 0x00fff000);
	if (drawctrl0 != dev_priv->state.s4.drawctrl0 ||
	    drawctrl1 != dev_priv->state.s4.drawctrl1) {
		DMA_LOCALS;
		BEGIN_DMA(4);
		DMA_WRITE(BCI_CMD_WAIT|BCI_CMD_WAIT_3D);
		DMA_SET_REGISTERS(SAVAGE_DRAWCTRL0_S4, 2);
		DMA_WRITE(drawctrl0);
		DMA_WRITE(drawctrl1);
		dev_priv->state.s4.drawctrl0 = drawctrl0;
		dev_priv->state.s4.drawctrl1 = drawctrl1;
		dev_priv->waiting = 1;
		DMA_COMMIT();
	}
}

static int savage_verify_texaddr(drm_savage_private_t *dev_priv, int unit,
				 uint32_t addr)
{
	if ((addr & 6) != 2) { /* reserved bits */
		DRM_ERROR("bad texAddr%d %08x (reserved bits)\n", unit, addr);
		return DRM_ERR(EINVAL);
	}
	if (!(addr & 1)) { /* local */
		addr &= ~7;
		if (addr <  dev_priv->texture_offset ||
		    addr >= dev_priv->texture_offset+dev_priv->texture_size) {
			DRM_ERROR
			    ("bad texAddr%d %08x (local addr out of range)\n",
			     unit, addr);
			return DRM_ERR(EINVAL);
		}
	} else { /* AGP */
		if (!dev_priv->agp_textures) {
			DRM_ERROR("bad texAddr%d %08x (AGP not available)\n",
				  unit, addr);
			return DRM_ERR(EINVAL);
		}
		addr &= ~7;
		if (addr < dev_priv->agp_textures->offset ||
		    addr >= (dev_priv->agp_textures->offset +
			     dev_priv->agp_textures->size)) {
			DRM_ERROR
			    ("bad texAddr%d %08x (AGP addr out of range)\n",
			     unit, addr);
			return DRM_ERR(EINVAL);
		}
	}
	return 0;
}

#define SAVE_STATE(reg,where)			\
	if(start <= reg && start+count > reg)	\
		dev_priv->state.where = regs[reg - start]
#define SAVE_STATE_MASK(reg,where,mask) do {			\
	if(start <= reg && start+count > reg) {			\
		uint32_t tmp;					\
		tmp = regs[reg - start];			\
		dev_priv->state.where = (tmp & (mask)) |	\
			(dev_priv->state.where & ~(mask));	\
	}							\
} while (0)
static int savage_verify_state_s3d(drm_savage_private_t *dev_priv,
				   unsigned int start, unsigned int count,
				   const uint32_t *regs)
{
	if (start < SAVAGE_TEXPALADDR_S3D ||
	    start+count-1 > SAVAGE_DESTTEXRWWATERMARK_S3D) {
		DRM_ERROR("invalid register range (0x%04x-0x%04x)\n",
			  start, start+count-1);
		return DRM_ERR(EINVAL);
	}

	SAVE_STATE_MASK(SAVAGE_SCSTART_S3D, s3d.new_scstart,
			~SAVAGE_SCISSOR_MASK_S3D);
	SAVE_STATE_MASK(SAVAGE_SCEND_S3D, s3d.new_scend,
			~SAVAGE_SCISSOR_MASK_S3D);

	/* if any texture regs were changed ... */
	if (start <= SAVAGE_TEXCTRL_S3D &&
	    start+count > SAVAGE_TEXPALADDR_S3D) {
		/* ... check texture state */
		SAVE_STATE(SAVAGE_TEXCTRL_S3D, s3d.texctrl);
		SAVE_STATE(SAVAGE_TEXADDR_S3D, s3d.texaddr);
		if (dev_priv->state.s3d.texctrl & SAVAGE_TEXCTRL_TEXEN_MASK)
			return savage_verify_texaddr(dev_priv, 0,
						dev_priv->state.s3d.texaddr);
	}

	return 0;
}

static int savage_verify_state_s4(drm_savage_private_t *dev_priv,
				  unsigned int start, unsigned int count,
				  const uint32_t *regs)
{
	int ret = 0;

	if (start < SAVAGE_DRAWLOCALCTRL_S4 ||
	    start+count-1 > SAVAGE_TEXBLENDCOLOR_S4) {
		DRM_ERROR("invalid register range (0x%04x-0x%04x)\n",
			  start, start+count-1);
		return DRM_ERR(EINVAL);
	}

	SAVE_STATE_MASK(SAVAGE_DRAWCTRL0_S4, s4.new_drawctrl0,
			~SAVAGE_SCISSOR_MASK_S4);
	SAVE_STATE_MASK(SAVAGE_DRAWCTRL1_S4, s4.new_drawctrl1,
			~SAVAGE_SCISSOR_MASK_S4);

	/* if any texture regs were changed ... */
	if (start <= SAVAGE_TEXDESCR_S4 &&
	    start + count > SAVAGE_TEXPALADDR_S4) {
		/* ... check texture state */
		SAVE_STATE(SAVAGE_TEXDESCR_S4, s4.texdescr);
		SAVE_STATE(SAVAGE_TEXADDR0_S4, s4.texaddr0);
		SAVE_STATE(SAVAGE_TEXADDR1_S4, s4.texaddr1);
		if (dev_priv->state.s4.texdescr & SAVAGE_TEXDESCR_TEX0EN_MASK)
			ret |= savage_verify_texaddr(dev_priv, 0,
						dev_priv->state.s4.texaddr0);
		if (dev_priv->state.s4.texdescr & SAVAGE_TEXDESCR_TEX1EN_MASK)
			ret |= savage_verify_texaddr(dev_priv, 1,
						dev_priv->state.s4.texaddr1);
	}

	return ret;
}
#undef SAVE_STATE
#undef SAVE_STATE_MASK

static int savage_dispatch_state(drm_savage_private_t *dev_priv,
				 const drm_savage_cmd_header_t *cmd_header,
				 const uint32_t *regs)
{
	unsigned int count = cmd_header->state.count;
	unsigned int start = cmd_header->state.start;
	unsigned int count2 = 0;
	unsigned int bci_size;
	int ret;
	DMA_LOCALS;

	if (!count)
		return 0;

	if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
		ret = savage_verify_state_s3d(dev_priv, start, count, regs);
		if (ret != 0)
			return ret;
		/* scissor regs are emitted in savage_dispatch_draw */
		if (start < SAVAGE_SCSTART_S3D) {
			if (start+count > SAVAGE_SCEND_S3D+1)
				count2 = count - (SAVAGE_SCEND_S3D+1 - start);
			if (start+count > SAVAGE_SCSTART_S3D)
				count = SAVAGE_SCSTART_S3D - start;
		} else if (start <= SAVAGE_SCEND_S3D) {
			if (start+count > SAVAGE_SCEND_S3D+1) {
				count -= SAVAGE_SCEND_S3D+1 - start;
				start = SAVAGE_SCEND_S3D+1;
			} else
				return 0;
		}
	} else {
		ret = savage_verify_state_s4(dev_priv, start, count, regs);
		if (ret != 0)
			return ret;
		/* scissor regs are emitted in savage_dispatch_draw */
		if (start < SAVAGE_DRAWCTRL0_S4) {
			if (start+count > SAVAGE_DRAWCTRL1_S4+1)
				count2 = count -
					 (SAVAGE_DRAWCTRL1_S4 + 1 - start);
			if (start+count > SAVAGE_DRAWCTRL0_S4)
				count = SAVAGE_DRAWCTRL0_S4 - start;
		} else if (start <= SAVAGE_DRAWCTRL1_S4) {
			if (start+count > SAVAGE_DRAWCTRL1_S4+1) {
				count -= SAVAGE_DRAWCTRL1_S4+1 - start;
				start = SAVAGE_DRAWCTRL1_S4+1;
			} else
				return 0;
		}
	}

	bci_size = count + (count+254)/255 + count2 + (count2+254)/255;

	if (cmd_header->state.global) {
		BEGIN_DMA(bci_size+1);
		DMA_WRITE(BCI_CMD_WAIT | BCI_CMD_WAIT_3D);
		dev_priv->waiting = 1;
	} else {
		BEGIN_DMA(bci_size);
	}

	do {
		while (count > 0) {
			unsigned int n = count < 255 ? count : 255;
			DMA_SET_REGISTERS(start, n);
			DMA_COPY(regs, n);
			count -= n;
			start += n;
			regs += n;
		}
		start += 2;
		regs += 2;
		count = count2;
		count2 = 0;
	} while (count);

	DMA_COMMIT();

	return 0;
}

static int savage_dispatch_dma_prim(drm_savage_private_t *dev_priv,
				    const drm_savage_cmd_header_t *cmd_header,
				    const drm_buf_t *dmabuf)
{
	unsigned char reorder = 0;
	unsigned int prim = cmd_header->prim.prim;
	unsigned int skip = cmd_header->prim.skip;
	unsigned int n = cmd_header->prim.count;
	unsigned int start = cmd_header->prim.start;
	unsigned int i;
	BCI_LOCALS;

	if (!dmabuf) {
	    DRM_ERROR("called without dma buffers!\n");
	    return DRM_ERR(EINVAL);
	}

	if (!n)
		return 0;

	switch (prim) {
	case SAVAGE_PRIM_TRILIST_201:
		reorder = 1;
		prim = SAVAGE_PRIM_TRILIST;
	case SAVAGE_PRIM_TRILIST:
		if (n % 3 != 0) {
			DRM_ERROR("wrong number of vertices %u in TRILIST\n",
				  n);
			return DRM_ERR(EINVAL);
		}
		break;
	case SAVAGE_PRIM_TRISTRIP:
	case SAVAGE_PRIM_TRIFAN:
		if (n < 3) {
			DRM_ERROR
			   ("wrong number of vertices %u in TRIFAN/STRIP\n",
			    n);
			return DRM_ERR(EINVAL);
		}
		break;
	default:
		DRM_ERROR("invalid primitive type %u\n", prim);
		return DRM_ERR(EINVAL);
	}

	if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
		if (skip != 0) {
			DRM_ERROR("invalid skip flags 0x%04x for DMA\n", skip);
			return DRM_ERR(EINVAL);
		}
	} else {
		unsigned int size = 10 - (skip & 1) - (skip >> 1 & 1) -
			(skip >> 2 & 1) - (skip >> 3 & 1) - (skip >> 4 & 1) -
			(skip >> 5 & 1) - (skip >> 6 & 1) - (skip >> 7 & 1);
		if (skip > SAVAGE_SKIP_ALL_S4 || size != 8) {
			DRM_ERROR("invalid skip flags 0x%04x for DMA\n", skip);
			return DRM_ERR(EINVAL);
		}
		if (reorder) {
			DRM_ERROR("TRILIST_201 used on Savage4 hardware\n");
			return DRM_ERR(EINVAL);
		}
	}

	if (start + n > dmabuf->total/32) {
		DRM_ERROR("vertex indices (%u-%u) out of range (0-%u)\n",
			  start, start + n - 1, dmabuf->total/32);
		return DRM_ERR(EINVAL);
	}

	/* Vertex DMA doesn't work with command DMA at the same time,
	 * so we use BCI_... to submit commands here. Flush buffered
	 * faked DMA first. */
	DMA_FLUSH();

	if (dmabuf->bus_address != dev_priv->state.common.vbaddr) {
		BEGIN_BCI(2);
		BCI_SET_REGISTERS(SAVAGE_VERTBUFADDR, 1);
		BCI_WRITE(dmabuf->bus_address | dev_priv->dma_type);
		dev_priv->state.common.vbaddr = dmabuf->bus_address;
	}
	if (S3_SAVAGE3D_SERIES(dev_priv->chipset) && dev_priv->waiting) {
		/* Workaround for what looks like a hardware bug. If a
		 * WAIT_3D_IDLE was emitted some time before the
		 * indexed drawing command then the engine will lock
		 * up. There are two known workarounds:
		 * WAIT_IDLE_EMPTY or emit at least 63 NOPs. */
		BEGIN_BCI(63);
		for (i = 0; i < 63; ++i)
			BCI_WRITE(BCI_CMD_WAIT);
		dev_priv->waiting = 0;
	}

	prim <<= 25;
	while (n != 0) {
		/* Can emit up to 255 indices (85 triangles) at once. */
		unsigned int count = n > 255 ? 255 : n;
		if (reorder) {
			/* Need to reorder indices for correct flat
			 * shading while preserving the clock sense
			 * for correct culling. Only on Savage3D. */
			int reorder[3] = {-1, -1, -1};
			reorder[start%3] = 2;

			BEGIN_BCI((count+1+1)/2);
			BCI_DRAW_INDICES_S3D(count, prim, start+2);

			for (i = start+1; i+1 < start+count; i += 2)
				BCI_WRITE((i + reorder[i % 3]) |
					  ((i + 1 +
					    reorder[(i + 1) % 3]) << 16));
			if (i < start+count)
				BCI_WRITE(i + reorder[i%3]);
		} else if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
			BEGIN_BCI((count+1+1)/2);
			BCI_DRAW_INDICES_S3D(count, prim, start);

			for (i = start+1; i+1 < start+count; i += 2)
				BCI_WRITE(i | ((i+1) << 16));
			if (i < start+count)
				BCI_WRITE(i);
		} else {
			BEGIN_BCI((count+2+1)/2);
			BCI_DRAW_INDICES_S4(count, prim, skip);

			for (i = start; i+1 < start+count; i += 2)
				BCI_WRITE(i | ((i+1) << 16));
			if (i < start+count)
				BCI_WRITE(i);
		}

		start += count;
		n -= count;

		prim |= BCI_CMD_DRAW_CONT;
	}

	return 0;
}

static int savage_dispatch_vb_prim(drm_savage_private_t *dev_priv,
				   const drm_savage_cmd_header_t *cmd_header,
				   const uint32_t *vtxbuf, unsigned int vb_size,
				   unsigned int vb_stride)
{
	unsigned char reorder = 0;
	unsigned int prim = cmd_header->prim.prim;
	unsigned int skip = cmd_header->prim.skip;
	unsigned int n = cmd_header->prim.count;
	unsigned int start = cmd_header->prim.start;
	unsigned int vtx_size;
	unsigned int i;
	DMA_LOCALS;

	if (!n)
		return 0;

	switch (prim) {
	case SAVAGE_PRIM_TRILIST_201:
		reorder = 1;
		prim = SAVAGE_PRIM_TRILIST;
	case SAVAGE_PRIM_TRILIST:
		if (n % 3 != 0) {
			DRM_ERROR("wrong number of vertices %u in TRILIST\n",
				  n);
			return DRM_ERR(EINVAL);
		}
		break;
	case SAVAGE_PRIM_TRISTRIP:
	case SAVAGE_PRIM_TRIFAN:
		if (n < 3) {
			DRM_ERROR
			    ("wrong number of vertices %u in TRIFAN/STRIP\n",
			     n);
			return DRM_ERR(EINVAL);
		}
		break;
	default:
		DRM_ERROR("invalid primitive type %u\n", prim);
		return DRM_ERR(EINVAL);
	}

	if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
		if (skip > SAVAGE_SKIP_ALL_S3D) {
			DRM_ERROR("invalid skip flags 0x%04x\n", skip);
			return DRM_ERR(EINVAL);
		}
		vtx_size = 8; /* full vertex */
	} else {
		if (skip > SAVAGE_SKIP_ALL_S4) {
			DRM_ERROR("invalid skip flags 0x%04x\n", skip);
			return DRM_ERR(EINVAL);
		}
		vtx_size = 10; /* full vertex */
	}

	vtx_size -= (skip & 1) + (skip >> 1 & 1) +
		(skip >> 2 & 1) + (skip >> 3 & 1) + (skip >> 4 & 1) +
		(skip >> 5 & 1) + (skip >> 6 & 1) + (skip >> 7 & 1);

	if (vtx_size > vb_stride) {
		DRM_ERROR("vertex size greater than vb stride (%u > %u)\n",
			  vtx_size, vb_stride);
		return DRM_ERR(EINVAL);
	}

	if (start + n > vb_size / (vb_stride*4)) {
		DRM_ERROR("vertex indices (%u-%u) out of range (0-%u)\n",
			  start, start + n - 1, vb_size / (vb_stride*4));
		return DRM_ERR(EINVAL);
	}

	prim <<= 25;
	while (n != 0) {
		/* Can emit up to 255 vertices (85 triangles) at once. */
		unsigned int count = n > 255 ? 255 : n;
		if (reorder) {
			/* Need to reorder vertices for correct flat
			 * shading while preserving the clock sense
			 * for correct culling. Only on Savage3D. */
			int reorder[3] = {-1, -1, -1};
			reorder[start%3] = 2;

			BEGIN_DMA(count*vtx_size+1);
			DMA_DRAW_PRIMITIVE(count, prim, skip);

			for (i = start; i < start+count; ++i) {
				unsigned int j = i + reorder[i % 3];
				DMA_COPY(&vtxbuf[vb_stride*j], vtx_size);
			}

			DMA_COMMIT();
		} else {
			BEGIN_DMA(count*vtx_size+1);
			DMA_DRAW_PRIMITIVE(count, prim, skip);

			if (vb_stride == vtx_size) {
				DMA_COPY(&vtxbuf[vb_stride*start],
					 vtx_size*count);
			} else {
				for (i = start; i < start+count; ++i) {
					DMA_COPY(&vtxbuf[vb_stride*i],
						 vtx_size);
				}
			}

			DMA_COMMIT();
		}

		start += count;
		n -= count;

		prim |= BCI_CMD_DRAW_CONT;
	}

	return 0;
}

static int savage_dispatch_dma_idx(drm_savage_private_t *dev_priv,
				   const drm_savage_cmd_header_t *cmd_header,
				   const uint16_t *idx,
				   const drm_buf_t *dmabuf)
{
	unsigned char reorder = 0;
	unsigned int prim = cmd_header->idx.prim;
	unsigned int skip = cmd_header->idx.skip;
	unsigned int n = cmd_header->idx.count;
	unsigned int i;
	BCI_LOCALS;

	if (!dmabuf) {
	    DRM_ERROR("called without dma buffers!\n");
	    return DRM_ERR(EINVAL);
	}

	if (!n)
		return 0;

	switch (prim) {
	case SAVAGE_PRIM_TRILIST_201:
		reorder = 1;
		prim = SAVAGE_PRIM_TRILIST;
	case SAVAGE_PRIM_TRILIST:
		if (n % 3 != 0) {
			DRM_ERROR("wrong number of indices %u in TRILIST\n", n);
			return DRM_ERR(EINVAL);
		}
		break;
	case SAVAGE_PRIM_TRISTRIP:
	case SAVAGE_PRIM_TRIFAN:
		if (n < 3) {
			DRM_ERROR
			    ("wrong number of indices %u in TRIFAN/STRIP\n", n);
			return DRM_ERR(EINVAL);
		}
		break;
	default:
		DRM_ERROR("invalid primitive type %u\n", prim);
		return DRM_ERR(EINVAL);
	}

	if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
		if (skip != 0) {
			DRM_ERROR("invalid skip flags 0x%04x for DMA\n", skip);
			return DRM_ERR(EINVAL);
		}
	} else {
		unsigned int size = 10 - (skip & 1) - (skip >> 1 & 1) -
			(skip >> 2 & 1) - (skip >> 3 & 1) - (skip >> 4 & 1) -
			(skip >> 5 & 1) - (skip >> 6 & 1) - (skip >> 7 & 1);
		if (skip > SAVAGE_SKIP_ALL_S4 || size != 8) {
			DRM_ERROR("invalid skip flags 0x%04x for DMA\n", skip);
			return DRM_ERR(EINVAL);
		}
		if (reorder) {
			DRM_ERROR("TRILIST_201 used on Savage4 hardware\n");
			return DRM_ERR(EINVAL);
		}
	}

	/* Vertex DMA doesn't work with command DMA at the same time,
	 * so we use BCI_... to submit commands here. Flush buffered
	 * faked DMA first. */
	DMA_FLUSH();

	if (dmabuf->bus_address != dev_priv->state.common.vbaddr) {
		BEGIN_BCI(2);
		BCI_SET_REGISTERS(SAVAGE_VERTBUFADDR, 1);
		BCI_WRITE(dmabuf->bus_address | dev_priv->dma_type);
		dev_priv->state.common.vbaddr = dmabuf->bus_address;
	}
	if (S3_SAVAGE3D_SERIES(dev_priv->chipset) && dev_priv->waiting) {
		/* Workaround for what looks like a hardware bug. If a
		 * WAIT_3D_IDLE was emitted some time before the
		 * indexed drawing command then the engine will lock
		 * up. There are two known workarounds:
		 * WAIT_IDLE_EMPTY or emit at least 63 NOPs. */
		BEGIN_BCI(63);
		for (i = 0; i < 63; ++i)
			BCI_WRITE(BCI_CMD_WAIT);
		dev_priv->waiting = 0;
	}

	prim <<= 25;
	while (n != 0) {
		/* Can emit up to 255 indices (85 triangles) at once. */
		unsigned int count = n > 255 ? 255 : n;

		/* check indices */
		for (i = 0; i < count; ++i) {
			if (idx[i] > dmabuf->total/32) {
				DRM_ERROR("idx[%u]=%u out of range (0-%u)\n",
					  i, idx[i], dmabuf->total/32);
				return DRM_ERR(EINVAL);
			}
		}

		if (reorder) {
			/* Need to reorder indices for correct flat
			 * shading while preserving the clock sense
			 * for correct culling. Only on Savage3D. */
			int reorder[3] = {2, -1, -1};

			BEGIN_BCI((count+1+1)/2);
			BCI_DRAW_INDICES_S3D(count, prim, idx[2]);

			for (i = 1; i+1 < count; i += 2)
				BCI_WRITE(idx[i + reorder[i % 3]] |
					  (idx[i + 1 +
					   reorder[(i + 1) % 3]] << 16));
			if (i < count)
				BCI_WRITE(idx[i + reorder[i%3]]);
		} else if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
			BEGIN_BCI((count+1+1)/2);
			BCI_DRAW_INDICES_S3D(count, prim, idx[0]);

			for (i = 1; i+1 < count; i += 2)
				BCI_WRITE(idx[i] | (idx[i+1] << 16));
			if (i < count)
				BCI_WRITE(idx[i]);
		} else {
			BEGIN_BCI((count+2+1)/2);
			BCI_DRAW_INDICES_S4(count, prim, skip);

			for (i = 0; i+1 < count; i += 2)
				BCI_WRITE(idx[i] | (idx[i+1] << 16));
			if (i < count)
				BCI_WRITE(idx[i]);
		}

		idx += count;
		n -= count;

		prim |= BCI_CMD_DRAW_CONT;
	}

	return 0;
}

static int savage_dispatch_vb_idx(drm_savage_private_t *dev_priv,
				  const drm_savage_cmd_header_t *cmd_header,
				  const uint16_t *idx,
				  const uint32_t *vtxbuf,
				  unsigned int vb_size, unsigned int vb_stride)
{
	unsigned char reorder = 0;
	unsigned int prim = cmd_header->idx.prim;
	unsigned int skip = cmd_header->idx.skip;
	unsigned int n = cmd_header->idx.count;
	unsigned int vtx_size;
	unsigned int i;
	DMA_LOCALS;

	if (!n)
		return 0;

	switch (prim) {
	case SAVAGE_PRIM_TRILIST_201:
		reorder = 1;
		prim = SAVAGE_PRIM_TRILIST;
	case SAVAGE_PRIM_TRILIST:
		if (n % 3 != 0) {
			DRM_ERROR("wrong number of indices %u in TRILIST\n", n);
			return DRM_ERR(EINVAL);
		}
		break;
	case SAVAGE_PRIM_TRISTRIP:
	case SAVAGE_PRIM_TRIFAN:
		if (n < 3) {
			DRM_ERROR
			    ("wrong number of indices %u in TRIFAN/STRIP\n", n);
			return DRM_ERR(EINVAL);
		}
		break;
	default:
		DRM_ERROR("invalid primitive type %u\n", prim);
		return DRM_ERR(EINVAL);
	}

	if (S3_SAVAGE3D_SERIES(dev_priv->chipset)) {
		if (skip > SAVAGE_SKIP_ALL_S3D) {
			DRM_ERROR("invalid skip flags 0x%04x\n", skip);
			return DRM_ERR(EINVAL);
		}
		vtx_size = 8; /* full vertex */
	} else {
		if (skip > SAVAGE_SKIP_ALL_S4) {
			DRM_ERROR("invalid skip flags 0x%04x\n", skip);
			return DRM_ERR(EINVAL);
		}
		vtx_size = 10; /* full vertex */
	}

	vtx_size -= (skip & 1) + (skip >> 1 & 1) +
		(skip >> 2 & 1) + (skip >> 3 & 1) + (skip >> 4 & 1) +
		(skip >> 5 & 1) + (skip >> 6 & 1) + (skip >> 7 & 1);

	if (vtx_size > vb_stride) {
		DRM_ERROR("vertex size greater than vb stride (%u > %u)\n",
			  vtx_size, vb_stride);
		return DRM_ERR(EINVAL);
	}

	prim <<= 25;
	while (n != 0) {
		/* Can emit up to 255 vertices (85 triangles) at once. */
		unsigned int count = n > 255 ? 255 : n;

		/* Check indices */
		for (i = 0; i < count; ++i) {
			if (idx[i] > vb_size / (vb_stride*4)) {
				DRM_ERROR("idx[%u]=%u out of range (0-%u)\n",
					  i, idx[i],  vb_size / (vb_stride*4));
				return DRM_ERR(EINVAL);
			}
		}

		if (reorder) {
			/* Need to reorder vertices for correct flat
			 * shading while preserving the clock sense
			 * for correct culling. Only on Savage3D. */
			int reorder[3] = {2, -1, -1};

			BEGIN_DMA(count*vtx_size+1);
			DMA_DRAW_PRIMITIVE(count, prim, skip);

			for (i = 0; i < count; ++i) {
				unsigned int j = idx[i + reorder[i % 3]];
				DMA_COPY(&vtxbuf[vb_stride*j], vtx_size);
			}

			DMA_COMMIT();
		} else {
			BEGIN_DMA(count*vtx_size+1);
			DMA_DRAW_PRIMITIVE(count, prim, skip);

			for (i = 0; i < count; ++i) {
				unsigned int j = idx[i];
				DMA_COPY(&vtxbuf[vb_stride*j], vtx_size);
			}

			DMA_COMMIT();
		}

		idx += count;
		n -= count;

		prim |= BCI_CMD_DRAW_CONT;
	}

	return 0;
}

static int savage_dispatch_clear(drm_savage_private_t *dev_priv,
				 const drm_savage_cmd_header_t *cmd_header,
				 const drm_savage_cmd_header_t *data,
				 unsigned int nbox,
				 const drm_clip_rect_t *boxes)
{
	unsigned int flags = cmd_header->clear0.flags;
	unsigned int clear_cmd;
	unsigned int i, nbufs;
	DMA_LOCALS;

	if (nbox == 0)
		return 0;

	clear_cmd = BCI_CMD_RECT | BCI_CMD_RECT_XP | BCI_CMD_RECT_YP |
		BCI_CMD_SEND_COLOR | BCI_CMD_DEST_PBD_NEW;
	BCI_CMD_SET_ROP(clear_cmd,0xCC);

	nbufs = ((flags & SAVAGE_FRONT) ? 1 : 0) +
	    ((flags & SAVAGE_BACK) ? 1 : 0) + ((flags & SAVAGE_DEPTH) ? 1 : 0);
	if (nbufs == 0)
		return 0;

	if (data->clear1.mask != 0xffffffff) {
		/* set mask */
		BEGIN_DMA(2);
		DMA_SET_REGISTERS(SAVAGE_BITPLANEWTMASK, 1);
		DMA_WRITE(data->clear1.mask);
		DMA_COMMIT();
	}
	for (i = 0; i < nbox; ++i) {
		unsigned int x, y, w, h;
		unsigned int buf;

		x = boxes[i].x1, y = boxes[i].y1;
		w = boxes[i].x2 - boxes[i].x1;
		h = boxes[i].y2 - boxes[i].y1;
		BEGIN_DMA(nbufs*6);
		for (buf = SAVAGE_FRONT; buf <= SAVAGE_DEPTH; buf <<= 1) {
			if (!(flags & buf))
				continue;
			DMA_WRITE(clear_cmd);
			switch(buf) {
			case SAVAGE_FRONT:
				DMA_WRITE(dev_priv->front_offset);
				DMA_WRITE(dev_priv->front_bd);
				break;
			case SAVAGE_BACK:
				DMA_WRITE(dev_priv->back_offset);
				DMA_WRITE(dev_priv->back_bd);
				break;
			case SAVAGE_DEPTH:
				DMA_WRITE(dev_priv->depth_offset);
				DMA_WRITE(dev_priv->depth_bd);
				break;
			}
			DMA_WRITE(data->clear1.value);
			DMA_WRITE(BCI_X_Y(x, y));
			DMA_WRITE(BCI_W_H(w, h));
		}
		DMA_COMMIT();
	}
	if (data->clear1.mask != 0xffffffff) {
		/* reset mask */
		BEGIN_DMA(2);
		DMA_SET_REGISTERS(SAVAGE_BITPLANEWTMASK, 1);
		DMA_WRITE(0xffffffff);
		DMA_COMMIT();
	}

	return 0;
}

static int savage_dispatch_swap(drm_savage_private_t *dev_priv,
				unsigned int nbox, const drm_clip_rect_t *boxes)
{
	unsigned int swap_cmd;
	unsigned int i;
	DMA_LOCALS;

	if (nbox == 0)
		return 0;

	swap_cmd = BCI_CMD_RECT | BCI_CMD_RECT_XP | BCI_CMD_RECT_YP |
		BCI_CMD_SRC_PBD_COLOR_NEW | BCI_CMD_DEST_GBD;
	BCI_CMD_SET_ROP(swap_cmd,0xCC);

	for (i = 0; i < nbox; ++i) {
		BEGIN_DMA(6);
		DMA_WRITE(swap_cmd);
		DMA_WRITE(dev_priv->back_offset);
		DMA_WRITE(dev_priv->back_bd);
		DMA_WRITE(BCI_X_Y(boxes[i].x1, boxes[i].y1));
		DMA_WRITE(BCI_X_Y(boxes[i].x1, boxes[i].y1));
		DMA_WRITE(BCI_W_H(boxes[i].x2-boxes[i].x1,
				  boxes[i].y2-boxes[i].y1));
		DMA_COMMIT();
	}

	return 0;
}

static int savage_dispatch_draw(drm_savage_private_t *dev_priv,
				const drm_savage_cmd_header_t *start,
				const drm_savage_cmd_header_t *end,
				const drm_buf_t *dmabuf,
				const unsigned int *vtxbuf,
				unsigned int vb_size, unsigned int vb_stride,
				unsigned int nbox,
				const drm_clip_rect_t *boxes)
{
	unsigned int i, j;
	int ret;

	for (i = 0; i < nbox; ++i) {
		const drm_savage_cmd_header_t *cmdbuf;
		dev_priv->emit_clip_rect(dev_priv, &boxes[i]);

		cmdbuf = start;
		while (cmdbuf < end) {
			drm_savage_cmd_header_t cmd_header;
			cmd_header = *cmdbuf;
			cmdbuf++;
			switch (cmd_header.cmd.cmd) {
			case SAVAGE_CMD_DMA_PRIM:
				ret = savage_dispatch_dma_prim(
					dev_priv, &cmd_header, dmabuf);
				break;
			case SAVAGE_CMD_VB_PRIM:
				ret = savage_dispatch_vb_prim(
					dev_priv, &cmd_header,
					vtxbuf, vb_size, vb_stride);
				break;
			case SAVAGE_CMD_DMA_IDX:
				j = (cmd_header.idx.count + 3) / 4;
				/* j was check in savage_bci_cmdbuf */
				ret = savage_dispatch_dma_idx(dev_priv,
					&cmd_header, (const uint16_t *)cmdbuf,
					dmabuf);
				cmdbuf += j;
				break;
			case SAVAGE_CMD_VB_IDX:
				j = (cmd_header.idx.count + 3) / 4;
				/* j was check in savage_bci_cmdbuf */
				ret = savage_dispatch_vb_idx(dev_priv,
					&cmd_header, (const uint16_t *)cmdbuf,
					(const uint32_t *)vtxbuf, vb_size,
					vb_stride);
				cmdbuf += j;
				break;
			default:
				/* What's the best return code? EFAULT? */
				DRM_ERROR("IMPLEMENTATION ERROR: "
					  "non-drawing-command %d\n",
					  cmd_header.cmd.cmd);
				return DRM_ERR(EINVAL);
			}

			if (ret != 0)
				return ret;
		}
	}

	return 0;
}

int savage_bci_cmdbuf(DRM_IOCTL_ARGS)
{
	DRM_DEVICE;
	drm_savage_private_t *dev_priv = dev->dev_private;
	drm_device_dma_t *dma = dev->dma;
	drm_buf_t *dmabuf;
	drm_savage_cmdbuf_t cmdbuf;
	drm_savage_cmd_header_t *kcmd_addr = NULL;
	drm_savage_cmd_header_t *first_draw_cmd;
	unsigned int *kvb_addr = NULL;
	drm_clip_rect_t *kbox_addr = NULL;
	unsigned int i, j;
	int ret = 0;

	DRM_DEBUG("\n");
	
	LOCK_TEST_WITH_RETURN(dev, filp);

	DRM_COPY_FROM_USER_IOCTL(cmdbuf, (drm_savage_cmdbuf_t __user *)data,
				 sizeof(cmdbuf));

	if (dma && dma->buflist) {
		if (cmdbuf.dma_idx > dma->buf_count) {
			DRM_ERROR
			    ("vertex buffer index %u out of range (0-%u)\n",
			     cmdbuf.dma_idx, dma->buf_count-1);
			return DRM_ERR(EINVAL);
		}
		dmabuf = dma->buflist[cmdbuf.dma_idx];
	} else {
		dmabuf = NULL;
	}

	/* Copy the user buffers into kernel temporary areas.  This hasn't been
	 * a performance loss compared to VERIFYAREA_READ/
	 * COPY_FROM_USER_UNCHECKED when done in other drivers, and is correct
	 * for locking on FreeBSD.
	 */
	if (cmdbuf.size) {
		kcmd_addr = drm_alloc(cmdbuf.size * 8, DRM_MEM_DRIVER);
		if (kcmd_addr == NULL)
			return DRM_ERR(ENOMEM);

		if (DRM_COPY_FROM_USER(kcmd_addr, cmdbuf.cmd_addr,
				       cmdbuf.size * 8))
		{
			drm_free(kcmd_addr, cmdbuf.size * 8, DRM_MEM_DRIVER);
			return DRM_ERR(EFAULT);
		}
		cmdbuf.cmd_addr = kcmd_addr;
	}
	if (cmdbuf.vb_size) {
		kvb_addr = drm_alloc(cmdbuf.vb_size, DRM_MEM_DRIVER);
		if (kvb_addr == NULL) {
			ret = DRM_ERR(ENOMEM);
			goto done;
		}

		if (DRM_COPY_FROM_USER(kvb_addr, cmdbuf.vb_addr,
				       cmdbuf.vb_size)) {
			ret = DRM_ERR(EFAULT);
			goto done;
		}
		cmdbuf.vb_addr = kvb_addr;
	}
	if (cmdbuf.nbox) {
		kbox_addr = drm_alloc(cmdbuf.nbox * sizeof(drm_clip_rect_t),
				       DRM_MEM_DRIVER);
		if (kbox_addr == NULL) {
			ret = DRM_ERR(ENOMEM);
			goto done;
		}

		if (DRM_COPY_FROM_USER(kbox_addr, cmdbuf.box_addr,
				       cmdbuf.nbox * sizeof(drm_clip_rect_t))) {
			ret = DRM_ERR(EFAULT);
			goto done;
		}
		cmdbuf.box_addr = kbox_addr;
	}

	/* Make sure writes to DMA buffers are finished before sending
	 * DMA commands to the graphics hardware. */
	DRM_MEMORYBARRIER();

	/* Coming from user space. Don't know if the Xserver has
	 * emitted wait commands. Assuming the worst. */
	dev_priv->waiting = 1;

	i = 0;
	first_draw_cmd = NULL;
	while (i < cmdbuf.size) {
		drm_savage_cmd_header_t cmd_header;
		cmd_header = *(drm_savage_cmd_header_t *)cmdbuf.cmd_addr;
		cmdbuf.cmd_addr++;
		i++;

		/* Group drawing commands with same state to minimize
		 * iterations over clip rects. */
		j = 0;
		switch (cmd_header.cmd.cmd) {
		case SAVAGE_CMD_DMA_IDX:
		case SAVAGE_CMD_VB_IDX:
			j = (cmd_header.idx.count + 3) / 4;
			if (i + j > cmdbuf.size) {
				DRM_ERROR("indexed drawing command extends "
					  "beyond end of command buffer\n");
				DMA_FLUSH();
				return DRM_ERR(EINVAL);
			}
			/* fall through */
		case SAVAGE_CMD_DMA_PRIM:
		case SAVAGE_CMD_VB_PRIM:
			if (!first_draw_cmd)
				first_draw_cmd = cmdbuf.cmd_addr-1;
			cmdbuf.cmd_addr += j;
			i += j;
			break;
		default:
			if (first_draw_cmd) {
				ret = savage_dispatch_draw (
					dev_priv, first_draw_cmd,
					cmdbuf.cmd_addr-1,
					dmabuf, cmdbuf.vb_addr, cmdbuf.vb_size,
					cmdbuf.vb_stride,
					cmdbuf.nbox, cmdbuf.box_addr);
				if (ret != 0)
					return ret;
				first_draw_cmd = NULL;
			}
		}
		if (first_draw_cmd)
			continue;

		switch (cmd_header.cmd.cmd) {
		case SAVAGE_CMD_STATE:
			j = (cmd_header.state.count + 1) / 2;
			if (i + j > cmdbuf.size) {
				DRM_ERROR("command SAVAGE_CMD_STATE extends "
					  "beyond end of command buffer\n");
				DMA_FLUSH();
				ret = DRM_ERR(EINVAL);
				goto done;
			}
			ret = savage_dispatch_state(dev_priv, &cmd_header,
				(const uint32_t *)cmdbuf.cmd_addr);
			cmdbuf.cmd_addr += j;
			i += j;
			break;
		case SAVAGE_CMD_CLEAR:
			if (i + 1 > cmdbuf.size) {
				DRM_ERROR("command SAVAGE_CMD_CLEAR extends "
					  "beyond end of command buffer\n");
				DMA_FLUSH();
				ret = DRM_ERR(EINVAL);
				goto done;
			}
			ret = savage_dispatch_clear(dev_priv, &cmd_header,
						    cmdbuf.cmd_addr,
						    cmdbuf.nbox, cmdbuf.box_addr);
			cmdbuf.cmd_addr++;
			i++;
			break;
		case SAVAGE_CMD_SWAP:
			ret = savage_dispatch_swap(dev_priv, cmdbuf.nbox,
						   cmdbuf.box_addr);
			break;
		default:
			DRM_ERROR("invalid command 0x%x\n", cmd_header.cmd.cmd);
			DMA_FLUSH();
			ret = DRM_ERR(EINVAL);
			goto done;
		}

		if (ret != 0) {
			DMA_FLUSH();
			goto done;
		}
	}

	if (first_draw_cmd) {
		ret = savage_dispatch_draw (
			dev_priv, first_draw_cmd, cmdbuf.cmd_addr, dmabuf,
			cmdbuf.vb_addr, cmdbuf.vb_size, cmdbuf.vb_stride,
			cmdbuf.nbox, cmdbuf.box_addr);
		if (ret != 0) {
			DMA_FLUSH();
			goto done;
		}
	}

	DMA_FLUSH();

	if (dmabuf && cmdbuf.discard) {
		drm_savage_buf_priv_t *buf_priv = dmabuf->dev_private;
		uint16_t event;
		event = savage_bci_emit_event(dev_priv, SAVAGE_WAIT_3D);
		SET_AGE(&buf_priv->age, event, dev_priv->event_wrap);
		savage_freelist_put(dev, dmabuf);
	}

done:
	/* If we didn't need to allocate them, these'll be NULL */
	drm_free(kcmd_addr, cmdbuf.size * 8, DRM_MEM_DRIVER);
	drm_free(kvb_addr, cmdbuf.vb_size, DRM_MEM_DRIVER);
	drm_free(kbox_addr, cmdbuf.nbox * sizeof(drm_clip_rect_t),
		 DRM_MEM_DRIVER);

	return ret;
}
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-09 04:13:32 +0000