{
  "@context":[
    "https://www.w3.org/ns/activitystreams",
    {"Hashtag":"as:Hashtag"}
  ],
  "published":"2023-11-29T22:35:23.514Z",
  "attributedTo":"https://gopinath.org/actors/rahul",
  "to":["https://www.w3.org/ns/activitystreams#Public","https://wandering.shop/users/cstross"],
  "cc":["https://gopinath.org/actors/rahul/followers"],
  "content":"<p>Running LLMs from a single file! With <a href=\"https://hacks.mozilla.org/2023/11/introducing-llamafile/\">LLamaFile</a> from Mozilla. All you need is </p><blockquote>curl -LO https://huggingface.co/jartine/llava-v1.5-7B-GGUF/resolve/main/llamafile-server-0.1-llava-v1.5-7b-q4<br>chmod 755 llamafile-server-0.1-llava-v1.5-7b-q4<br>./llamafile-server-0.1-llava-v1.5-7b-q4</blockquote><p><br>Now, I am waiting for an easy way to finetune an LLM from a Mac (this is <a href=\"https://github.com/ggerganov/llama.cpp/tree/master/examples/finetune\">mostly doable now</a>) or using a CPU, and we will soon be within reach of the world of custom agents promised by <a href=\"https://wandering.shop/users/cstross\" class=\"mention\" rel=\"tag\">@cstross</a></p>",
  "mediaType":"text/html",
  "attachment":[],
  "tag":[
      {"type":"Mention","name":"@cstross@wandering.shop","href":"https://wandering.shop/users/cstross"}
  ],
  "type":"Note",
  "id":"https://gopinath.org/objects/bIno66_QH3U"
}