Re: How to find RAM/disk usage of each vector field

2024-11-06 Thread Michael McCandless
On Tue, Nov 5, 2024 at 5:17 PM Adrien Grand wrote Why is it important to break down per field as opposed to scaling based on > the total volume of vector data? > It's really for internal planning purposes / service telemetry ... at Amazon product search team (where I also work w/ Tanmay -- hi Ta

Re: How to find RAM/disk usage of each vector field

2024-11-06 Thread Michael McCandless
On Tue, Nov 5, 2024 at 7:31 PM Patrick Zhai wrote: I wouldn't call this a good way, but as the last resort you can parse the > metadata files yourself, as it is not so hard to parse (yet) Yeah ... the Lucene codec itself knows precisely how much disk is used for each field, and indeed stores it