Much of the performance problem comes from layers on top of NTFS itself- it's not just the virus scanner. Ext4 might be faster but I doubt it would be enough to ditch WSL2 for those use cases that need it.
Also, some of the "performance problems" are simply different access models. Windows and NTFS tries to provide some database-like ACID characteristics, including transactions at the level of batches of file updates with commit/rollback support. Ext4 and Linux (intentionally) make few such guarantees and so it shouldn't be surprising have very different performance profiles, just as you might expect between a NoSQL database that makes no ACID guarantees and an SQL database with multiple types of locks and several types of transaction behaviors.
