vmsvga: correct bitmap and pixmap size checks

When processing svga command DEFINE_CURSOR in vmsvga_fifo_run,
the computed BITMAP and PIXMAP size are checked against the
'cursor.mask[]' and 'cursor.image[]' array sizes in bytes.
Correct these checks to avoid OOB memory access.

Reported-by: Qinghao Tang <luodalongde@gmail.com>
Reported-by: Li Qiang <liqiang6-s@360.cn>
Signed-off-by: Prasad J Pandit <pjp@fedoraproject.org>
Message-id: 1473338754-15430-1-git-send-email-ppandit@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

diff --git a/hw/display/vmware_vga.c b/hw/display/vmware_vga.c
index e51a05ea7e48505af029ae9e5e9b0a243812d1d0..6599cf078d22f794e9eb54450f602ce022136fe8 100644 (file)
--- a/hw/display/vmware_vga.c
+++ b/hw/display/vmware_vga.c
@@ -676,11 +676,13 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s)
             cursor.bpp = vmsvga_fifo_read(s);
 
             args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp);
-            if (cursor.width > 256 ||
-                cursor.height > 256 ||
-                cursor.bpp > 32 ||
-                SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask ||
-                SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image) {
+            if (cursor.width > 256
+                || cursor.height > 256
+                || cursor.bpp > 32
+                || SVGA_BITMAP_SIZE(x, y)
+                    > sizeof(cursor.mask) / sizeof(cursor.mask[0])
+                || SVGA_PIXMAP_SIZE(x, y, cursor.bpp)
+                    > sizeof(cursor.image) / sizeof(cursor.image[0])) {
                     goto badcmd;
             }