From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazon11020076.outbound.protection.outlook.com [52.101.69.76]) by lore.virtuozzo.com (Postfix) with ESMTPS id 85774802A5 for ; Wed, 30 Sep 2026 11:18:00 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pMyOeARsA9pXViFAepoVMQX7VOVncNNLvYzgVRPUIbIH1uNr/tbUIwYrtHXKvqOR+c4f7NeHbpsqdgHDdroOGifwkmBnkkzlIm1rUVxu94Kin2tWM2ITd5ni1CQzFfV2Pyqw+EakgM8S+DrH15BA0XfYU4BpKH4z45iiAUd298k39ZKH5n75U08etrJelKJAjs/DSaZzifjm9sjoG554XCKKF3fLTgzzEIOjTMQ9bcRyKVT3Fhe402jKd5WqCW9ufVKOYrnJwdwSYOzujDKf/g+FWtGFboKnwS6DiJoZCkM61QmA1Soic+HLMDDrAndagcCf5WChbO8JLDsLTejp8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ALOUmCAkHdGPF4huTU7Cg71JOGKZLPvjfbITX8q0tt0=; b=yDYk8dzqqrHy4Pt5X962xpEaTwB+BMvcM8nXhaAG18deiwPsFWQcMQml5xRax71h3i1HF9dTF/KsytSXMd5znrnywvI2cSJOlFaDak0KOBjV4/Q4tSHI2pQEIggxfavp2MMTC71X0VXuPh6DLNQn4PI9uvFtOg4KNKZ0iB7Rh9+9FzbZNscmElmhoHNbtcUZgw7VUOvj3UX1Ltqb286HjVYCpMc6jgIaFmQLMlPjvYnWZXu6f4IZ9boQQnBIr0O1nOhKiU9r7t/gwTkykIHfT6UPTW8In6qrucyS533x+hYZxnG4BE3vlDTYNSiezCHRMEprkbRjDrO7sbw1uvEe6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 2a00:1450:4864:31::10) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=openvz.org; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=openvz.org; dkim=pass (signature was verified) header.d=openvz.org; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ALOUmCAkHdGPF4huTU7Cg71JOGKZLPvjfbITX8q0tt0=; b=jktKkJkridyIyLHjtZ6cKwvzupGl2Ih+Pmpdgq5cCYkUWZUR+YzjJs5hanHFhwNOTXBzCj7+uWEgJftcA8qjKwvHxz9kSu8ok21b1DFKE4CxyM8DcL0UhUI2TGM+EX0GK61xQKi5RQAWGzG8bGErc8rezrFAFqGwpEiqpuS+8to= Received: from DUZPR01CA0077.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::15) by AS8PR08MB8898.eurprd08.prod.outlook.com (2603:10a6:20b:5b7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.496.9; Wed, 30 Sep 2026 11:17:55 +0000 Received: from DB1PEPF000509FA.eurprd03.prod.outlook.com (2603:10a6:10:46a:cafe::3a) by DUZPR01CA0077.outlook.office365.com (2603:10a6:10:46a::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.472.16 via Frontend Transport; Wed, 30 Sep 2026 11:17:55 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 2a00:1450:4864:31::10) smtp.mailfrom=openvz.org; dkim=pass (signature was verified) header.d=openvz.org;dmarc=pass action=none header.from=openvz.org; Received-SPF: Pass (protection.outlook.com: domain of openvz.org designates 2a00:1450:4864:31::10 as permitted sender) receiver=protection.outlook.com; client-ip=2a00:1450:4864:31::10; helo=mail-wm2-x10.google.com; pr=C Received: from mail-wm2-x10.google.com (2a00:1450:4864:31::10) by DB1PEPF000509FA.mail.protection.outlook.com (2603:10a6:18:3::6d4) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.472.14 via Frontend Transport; Wed, 30 Sep 2026 11:17:55 +0000 Received: by mail-wm2-x10.google.com with SMTP id 5b1f17b1804b1-49ffa15f67fso24801255e9.2 for ; Wed, 30 Sep 2026 04:17:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1790767074; x=1791371874; darn=virtuozzo.com; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ALOUmCAkHdGPF4huTU7Cg71JOGKZLPvjfbITX8q0tt0=; b=BcWbz063N8Ie4+gP9xxQDmOj/FSmh8YqmVeh6MwtJZBliEVX/PfzVx/Isf6+wCEk+2 b0npYg07qlGT879E3zQGF/698gfpmgtxKn6kkDwJfdMTes6XElqWR+0SK48MVRRiUkJT Yo/g4/AW7sBtlKto0zSloyV1lO18UmK8NHTvLxNz5O69mj2o3CHKMPNzIKyTalsid5cw Dpug+v8RTdpT9n6mbp6sDtaWnR8uXnH5jJUT1QceUSKpGHZuMosQ4MZs1c4KSGNR/q+5 a581oY9kocafuOJrOtl3KZtx1pIzZpeIjt9d4j/EJgIDz19qs6RasVoOB6ebBkeCi9mg wqng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790767074; x=1791371874; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ALOUmCAkHdGPF4huTU7Cg71JOGKZLPvjfbITX8q0tt0=; b=lPXJFvEQ2bNkCyNdfYFz+J/UXE5teVYM/YF1N0z7uDp/D1zqF909pGhVPq9bZblsJh xJeomIC+R9a/Tsj9sZ/IiwaRLsP2pxZ5nt6JyszsWEOMaO95tlbDkUvVCzZfTgIHeo8J BkfIUsufLyyrVzHCuEhS9QYlZ6jls5fX0tdf2KRkOnvx4wWKbQGm3zartkPe/MEY2FwG 8mDGhFH4Wb92AetLCqzQJGazeeYYRMJITvfbUkYVOCqOXoCoLKh01e4+W+4Wxh8oHxXg Z1yUCmlQt/GLWJnn/4LG78druZ88vKuy/n2DlAWkyUYWuATPm0c6fkIyUaw8DLLsvBww AIwA== X-Gm-Message-State: AFuF++lBcn8FRqvjEyHVfZseM7zMYl8yFEYYfX6m+r/K9+KPiSlNwx5I 5YwyOvLX0tmT9UZoLnePswEBy6FZHUMpcQAExT6HVM827mOsnsSkKtOWch3MDwIcnONCyK49xld PiCe0aHo= X-Gm-Gg: AYBFou3NqMGGDybPvuhyNthgAY0ZxQqfwyJGuJIwPEylxf3hyrVNQtgaD3txWOuWUQ4 jov5qDJSFEv+eHjsuyUDTjbicFjIPhjU8M8Qrly7aTvtA7CNsxDgzUkNCOt9y/2W95tCTPuHwcD Z00AXlY6LwJAxPrRfV1VsE1ZQelM9n+JX9guQ+VjBOuXZf0ThukssH7Vs0DA/gki8S2ed5evL/D 5htcsHRNp9zJ+zZD3zde0fNR4UWe2fN3ajwYFGyX/AhhSL0e4xbqwEMbuTXlAG/s/qJ93HQeYtH zjfIGi9tjoKZFHUnG0Cb9ihnJ05YFQGOawPV0k3UgEvsdx4GFg+2AfLYXs9QoNeEtNC5KX9Ooj8 xaAZEJA7Tgjs/UlMPerybxWjV2d/O5pgbktKdjLZ8+aT18J86H/6SEIQ++fDhANhaBcHyyhF4bZ q6J1u/a7gMtGIw8DDp4Zjwb7dBAZTJH/Q95LibPF0JAv8R3j6k4BgfQ/WZ51MIxWl5TPg= X-Received: by 2002:a05:600c:8b5b:b0:49f:ffd0:9a9a with SMTP id 5b1f17b1804b1-4a01b12055amr18074345e9.25.1790767074522; Wed, 30 Sep 2026 04:17:54 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:c6b8:98e2:93f0:2de6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4a01cd002bcsm3092365e9.4.2026.09.30.04.17.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Sep 2026 04:17:54 -0700 (PDT) From: "Denis V. Lunev" To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@virtuozzo.com Subject: [PATCH hci-8.0 7/9] block/block-copy: track known-zero source clusters #VSTOR-134085 Date: Wed, 30 Sep 2026 13:17:37 +0200 Message-ID: <20260930111739.3866427-8-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260930111739.3866427-1-den@openvz.org> References: <20260930111739.3866427-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509FA:EE_|AS8PR08MB8898:EE_ X-MS-Office365-Filtering-Correlation-Id: d785fa3b-07d1-4230-5739-08df1ee479a2 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 0 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|6059299003|376014|9140799003|82310400026|61400799027|48200799018|43022699015|10067099003|56012099006|5063699009|6133799003|55112099003|18002099003|22082099003|16102099003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?k0Pl5LQW4V0lmdgBlXiNN5KgXL68qy9lSGsdty5fUHaWKjWnRgTdNvzMvSN4?= =?us-ascii?Q?HAx3E2mlqQP99UqG6tohLjG/rcVJ87CHjMgtngZZ2STebzCxdbAhgrusFPMg?= =?us-ascii?Q?xEeuAJG+r/K+ieiHjlQldi64ahM9Ae452rMd2+DSy4CxhOh3NGDdip0FiNGt?= =?us-ascii?Q?2gyEMav/oNLLsZUx/KcKOQz/nwJV3gScTv9Yd6Gd0FSr8HPq5MQeaDaJCXkd?= =?us-ascii?Q?J8LfKPfGEfJi+u/DwNFjVMsKjtDz5Z/HoucPEDExIAHE8bV1vBjAGGnQyq8o?= =?us-ascii?Q?kxRjP0uanrqRMiey5WnqcuzqX20I3JLttcMVVOobHBGuZl0MzOB/rO7825Ub?= =?us-ascii?Q?gi96JSckM56BYsEt9TEfSD+KIlnVkYPJQGye5phIViP0+h/hzeo8bL1WApDB?= =?us-ascii?Q?UR7NBVHDP2vme4PG5RGiBD2QpGiRNJ11hKEi4FzgRTXChdDaoemBWMjqiJ2N?= =?us-ascii?Q?nmoTZCZ48v7XsEgIm/hj1hTWQvmimVsFT9TzGYRez4a8527TYND+Cj8pEu3j?= =?us-ascii?Q?1mDNgCb1SUwqOt0E+suoEosf4cKokHQD/n4IubwiV5dZDQ8h4g4FmcaZt5i7?= =?us-ascii?Q?oFTEusPEfM0U0bInfhbSbK385I26dvSOrFRTIs1ulqISwy+ftNUp17OLCORt?= =?us-ascii?Q?q3/Z2EdQ7XV1UjRevusmaktOTpeKQTCECh7lNQl4OyzRyPYy8MU2hZkZJIu6?= =?us-ascii?Q?EAWozHyUD380ZwpSh2U13UhYuJs7zyIpFNiHxZbflsqXG4XgyooStKDRXSeg?= =?us-ascii?Q?uGW+ZIxcD9jYkacXvAYBUceFWXMBYY514uT9ybTihub1eqfkBj3/eoooNVmz?= =?us-ascii?Q?nwRUMWP7/8s913poSHORZXVlxF0jjghRozioRI9olumC2u4Ke9J0M3cGJLt/?= =?us-ascii?Q?Tf6KvjeXxgk0Rh5Z4IGoSaDbgD+4YDlWX0KsIopaBgLsmPbKPfjS2jKX1qky?= =?us-ascii?Q?OzBF7aeR3yEl2p1IdR5W7o9reiBL75aim5Y8R6/NIqLqAMcbgdOoqWCZkLEu?= =?us-ascii?Q?1V0z+zbKBu5Ho3uCcDzIBRteNIj73cUnax2bAnUJEFg+Lrc=3D?= X-Forefront-Antispam-Report: CIP:2a00:1450:4864:31::10;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail-wm2-x10.google.com;PTR:mail-wm2-x10.google.com;CAT:NONE;SFS:(13230040)(6059299003)(376014)(9140799003)(82310400026)(61400799027)(48200799018)(43022699015)(10067099003)(56012099006)(5063699009)(6133799003)(55112099003)(18002099003)(22082099003)(16102099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: X1wkmT2ltZ8A/Gc9zPHRImjWg0jzi2ZmgogzvAoXt7zGJrlM5Xh+tb23oo7SIg5qcjiUpFgUh6LpUqzD5NQp1jSdd1OryNm1D/ZzlnUVV84zOo36D1A4OXrj/JRqVfAr/IXYcSFdLNJY6AeFLj+L+hLmvtB7YH9/7mPClciOySTgKpvXZFimtFuce6Nny5hwLESUOZSVyk3ZDJfwZejj258/OPB1IiAACVkhQg34ph75teXlNuCuGqELET6myCx2Ji/fJ49O9aqac22t5e6B1q+tk6DmpJ3VUovSDYiKxqcduDdBVmjYrfZypZVJo4wdg+oZYR/sV2njKqNhavrikfTsWiS2IecT01U7HnWWRdgzXMyQDrkkTZFC5F5OVxC/UKhYg+L859W7096iIN753uihHA7jqowKC/J9aZWb1RL3Nuqmq4G2TdYto6WAqwHb X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2026 11:17:55.1919 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d785fa3b-07d1-4230-5739-08df1ee479a2 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509FA.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8898 List-Id: From: Denis V. Lunev block_copy_reset_unallocated()'s up-front scan already queries the source, and the same query reports zero-ness. The copy loop re-queries per task and, having no answer up front, can only size a task by the copy buffer, so a mostly-zero image is cut into thousands of pieces that each turn out to read as zero. Cache the answer in zero_bitmap, published by zero_bitmap_valid once the scan is done. block_copy_task_create() then picks the method from the bitmap and ends the task where the answer changes, so a zero task never reaches into data and a copy task never swallows a zero run. The per-task query stays as the fallback until the flag is set, since CBW intercepts guest writes while the scan is still running. The scan and the query it replaces must resolve BDRV_BLOCK_ZERO against the same part of the chain, so the scan moves from bdrv_co_is_allocated() to bdrv_co_block_status_above() and the base selection is factored out into block_copy_status_base(). zero_bitmap is a plain HBitmap, not a BdrvDirtyBitmap: an internal cache has no business in query-named-block-nodes. One writer and readers gated by the flag need no mutex, but the publish needs ordering, hence store-release and load-acquire. It is allocated on demand, so sync=none and a standalone copy-before-write filter do not pay for it. hci-8.0: 6b1a4754713 ("backup: skip copying unallocated for full mode") scans for sync=full too and leaves skip_unallocated set after the scan, choosing the base from top_mode instead. So block_copy_status_base() keys on top_mode, and replaces the base block_copy_is_cluster_allocated() computed on its own. Reviewed-by: Andrey Drobyshev Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- block/backup.c | 1 + block/block-copy.c | 117 ++++++++++++++++++++++++++++++------- include/block/block-copy.h | 1 + 3 files changed, 98 insertions(+), 21 deletions(-) diff --git a/block/backup.c b/block/backup.c index f7d8421812a..03ad1ad350b 100644 --- a/block/backup.c +++ b/block/backup.c @@ -284,6 +284,7 @@ static int coroutine_fn backup_run(Job *job, Error **errp) offset += count; } + block_copy_set_zero_bitmap_valid(s->bcs); } if (s->sync_mode == MIRROR_SYNC_MODE_NONE) { diff --git a/block/block-copy.c b/block/block-copy.c index e210688b1df..27a5d1a41b2 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -20,6 +20,8 @@ #include "block/block_int-io.h" #include "block/dirty-bitmap.h" #include "block/reqlist.h" +#include "qemu/hbitmap.h" +#include "qemu/host-utils.h" #include "system/block-backend.h" #include "qemu/units.h" #include "qemu/co-shared-resource.h" @@ -159,7 +161,10 @@ typedef struct BlockCopyState { /* State fields that use a thread-safe API */ BdrvDirtyBitmap *copy_bitmap; bool top_mode; - + /* Clusters reading as zero; allocated on demand, frozen once valid. */ + HBitmap *zero_bitmap; + /* Published only once the scan is done. */ + bool zero_bitmap_valid; /* atomic, store-release/load-acquire */ ProgressMeter *progress; SharedResource *mem; RateLimit rate_limit; @@ -193,6 +198,7 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state, int64_t offset, int64_t bytes) { BlockCopyTask *task; + BlockCopyMethod method; int64_t max_chunk; QEMU_LOCK_GUARD(&s->lock); @@ -204,6 +210,27 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state, return NULL; } + method = s->method; + + /* + * The scan already knows how this range reads: pick the method here and + * stop the task where the answer changes. + */ + if (qatomic_load_acquire(&s->zero_bitmap_valid)) { + int64_t boundary; + + if (hbitmap_get(s->zero_bitmap, offset)) { + method = COPY_WRITE_ZEROES; + boundary = hbitmap_next_zero(s->zero_bitmap, offset, bytes); + } else { + boundary = hbitmap_next_dirty(s->zero_bitmap, offset, bytes); + } + + if (boundary >= 0) { + bytes = boundary - offset; + } + } + assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); bytes = QEMU_ALIGN_UP(bytes, s->cluster_size); @@ -218,7 +245,7 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state, .task.func = block_copy_task_entry, .s = s, .call_state = call_state, - .method = s->method, + .method = method, }; reqlist_init_req(&s->reqs, &task->req, offset, bytes); @@ -274,6 +301,9 @@ void block_copy_state_free(BlockCopyState *s) ratelimit_destroy(&s->rate_limit); bdrv_release_dirty_bitmap(s->copy_bitmap); + if (s->zero_bitmap) { + hbitmap_free(s->zero_bitmap); + } shres_destroy(s->mem); g_free(s); } @@ -655,20 +685,24 @@ static coroutine_fn int block_copy_task_entry(AioTask *task) return ret; } +/* The scan and the per-task query must resolve BDRV_BLOCK_ZERO alike. */ +static GRAPH_RDLOCK BlockDriverState *block_copy_status_base(BlockCopyState *s) +{ + if (qatomic_read(&s->top_mode)) { + return bdrv_backing_chain_next(s->source->bs); + } + + return NULL; +} + static coroutine_fn GRAPH_RDLOCK int block_copy_block_status(BlockCopyState *s, int64_t offset, int64_t bytes, int64_t *pnum) { int64_t num; - BlockDriverState *base; + BlockDriverState *base = block_copy_status_base(s); int ret; - if (qatomic_read(&s->top_mode)) { - base = bdrv_backing_chain_next(s->source->bs); - } else { - base = NULL; - } - ret = bdrv_co_block_status_above(s->source->bs, base, offset, bytes, &num, NULL, NULL); if (ret < 0 || num < s->cluster_size) { @@ -688,50 +722,79 @@ int block_copy_block_status(BlockCopyState *s, int64_t offset, int64_t bytes, return ret; } +/* Only the scan allocates, and it runs before zero_bitmap_valid. */ +static HBitmap *block_copy_zero_bitmap(BlockCopyState *s) +{ + if (!s->zero_bitmap) { + s->zero_bitmap = hbitmap_alloc(s->len, ctz32(s->cluster_size)); + } + + return s->zero_bitmap; +} + +static void block_copy_mark_zero_prefix(BlockCopyState *s, int64_t offset, + int64_t zero_count) +{ + int64_t zero_bytes = QEMU_ALIGN_DOWN(zero_count, s->cluster_size); + + if (zero_bytes > 0) { + hbitmap_set(block_copy_zero_bitmap(s), offset, zero_bytes); + } +} + /* * Check if the cluster starting at offset is allocated or not. * return via pnum the number of contiguous clusters sharing this allocation. + * Also marks the zero prefix of the range in zero_bitmap. */ static int coroutine_fn GRAPH_RDLOCK block_copy_is_cluster_allocated(BlockCopyState *s, int64_t offset, int64_t *pnum) { BlockDriverState *bs = s->source->bs; - BlockDriverState *base; + BlockDriverState *base = block_copy_status_base(s); + int64_t orig_offset = offset; int64_t count, total_count = 0; + int64_t zero_count = 0; + bool zero_broken = false; int64_t bytes = s->len - offset; int ret; - if (s->top_mode && s->source->bs->backing) { - base = s->source->bs->backing->bs; - } else { - base = NULL; - } - assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); while (true) { /* protected in backup_run() */ - ret = bdrv_co_is_allocated_above(bs, base, false, offset, bytes, - &count); + ret = bdrv_co_block_status_above(bs, base, offset, bytes, &count, + NULL, NULL); if (ret < 0) { return ret; } + if (!zero_broken) { + if (ret & BDRV_BLOCK_ZERO) { + zero_count += count; + } else { + zero_broken = true; + } + } + total_count += count; - if (ret || count == 0) { + if ((ret & BDRV_BLOCK_ALLOCATED) || count == 0) { /* - * ret: partial segment(s) are considered allocated. + * BDRV_BLOCK_ALLOCATED: partial segment(s) are considered + * allocated. * otherwise: unallocated tail is treated as an entire segment. */ *pnum = DIV_ROUND_UP(total_count, s->cluster_size); - return ret; + block_copy_mark_zero_prefix(s, orig_offset, zero_count); + return !!(ret & BDRV_BLOCK_ALLOCATED); } /* Unallocated segment(s) with uncertain following segment(s) */ if (total_count >= s->cluster_size) { *pnum = total_count / s->cluster_size; + block_copy_mark_zero_prefix(s, orig_offset, zero_count); return 0; } @@ -791,6 +854,12 @@ block_copy_set_task_method(BlockCopyState *s, BlockCopyTask *task) int ret; int64_t status_bytes; + /* block_copy_task_create() already decided, from zero_bitmap. */ + if (qatomic_load_acquire(&s->zero_bitmap_valid)) { + return true; + } + + /* CBW filter could call this early. */ ret = block_copy_block_status(s, task->req.offset, task->req.bytes, &status_bytes); assert(ret >= 0); /* never fail */ @@ -1122,6 +1191,12 @@ void block_copy_set_options(BlockCopyState *s, bool skip_unallocated, qatomic_set(&s->top_mode, top_mode); } +void block_copy_set_zero_bitmap_valid(BlockCopyState *s) +{ + block_copy_zero_bitmap(s); + qatomic_store_release(&s->zero_bitmap_valid, true); +} + void block_copy_set_speed(BlockCopyState *s, uint64_t speed) { ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME); diff --git a/include/block/block-copy.h b/include/block/block-copy.h index d9304d1cf56..57e13f82f78 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -102,5 +102,6 @@ int64_t block_copy_cluster_size(BlockCopyState *s); void block_copy_set_options(BlockCopyState *s, bool skip_unallocated, bool top_mode); void block_copy_set_global_error(BlockCopyState *s, int error); +void block_copy_set_zero_bitmap_valid(BlockCopyState *s); #endif /* BLOCK_COPY_H */ -- 2.53.0