Uv workflow improvements (#531)

* Uv workflow improvements

* Uv workflow improvements

* linter improvements

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* windows fixes

* windows fixes

* windows fixes

* windows fixes

* windows fixes

* windows fixes

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix
This commit is contained in:
Sebastian Raschka
2025-02-16 13:16:51 -06:00
committed by GitHub
parent 29353c74d8
commit a08d7aaa84
29 changed files with 382 additions and 321 deletions

View File

@@ -944,6 +944,7 @@
"## 9) Using PyTorch's FlexAttention\n",
"\n",
"- See [FlexAttention: The Flexibility of PyTorch with the Performance of FlashAttention](https://pytorch.org/blog/flexattention/) to learn more about FlexAttention\n",
"- FlexAttention caveat: It currently doesn't support dropout\n",
"- This is supported starting from PyTorch 2.5, which you can install on a CPU machine via\n",
"\n",
" ```bash\n",
@@ -1029,7 +1030,7 @@
" # (3, b, num_heads, num_tokens, head_dim) -> 3 times (b, num_heads, num_tokens, head_dim)\n",
" queries, keys, values = qkv\n",
"\n",
" use_dropout = 0. if not self.training else self.dropout\n",
" # use_dropout = 0. if not self.training else self.dropout\n",
"\n",
" # Ensure attn_mask is compatible with expected shape and `batch_first=True`\n",
" # No need to manually adjust for num_heads; ensure it's right for the sequence\n",
@@ -1967,7 +1968,7 @@
"provenance": []
},
"kernelspec": {
"display_name": "pt",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
@@ -1981,7 +1982,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
"version": "3.10.16"
}
},
"nbformat": 4,