Gregory's Server@webology thanks, updated that to say "Or the larger 90B model (55GB download, likely needs ~88GB of RAM) like this:"
|
Top-level
 @webology thanks, updated that to say "Or the larger 90B model (55GB download, likely needs ~88GB of RAM) like this:" 1 comment
|
@simon I hate that the memory size is so complicated. (that's my best rough guess btw)
Check out Matt's work https://www.youtube.com/@technovangelist
Check out this video https://www.youtube.com/watch?v=QfFRNF5AhME and one of Matt's about context windows. They are the best I have seen for explaining how much RAM one needs just to run one.
What annoys me is models are measured in Billions and context sizes in Kilobytes, and both are deceptive. i.e. 128K context size = 64 GB of RAM. NOT OBVIOUS at all.
@simon I hate that the memory size is so complicated. (that's my best rough guess btw)
Check out Matt's work https://www.youtube.com/@technovangelist
Check out this video https://www.youtube.com/watch?v=QfFRNF5AhME and one of Matt's about context windows. They are the best I have seen for explaining how much RAM one needs just to run one.