Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQL

<div class="post-cover"><div role="img" aria-label="Cover artwork: a wide, tight field of small grey dots. A large circular loupe magnifies part of the field, where the dots resolve into dark checks, crosses, and dashes — uniform from a distance, distinct under magnification. The loupe follows the mouse." data-cover-loupe="" data-config="{&quot;cx&quot;:1240,&quot;cy&quot;:500,&quot;r&quot;:232,&quot;k&quot;:2,&quot;w&quot;:1910,&quot;h&quot;:1000,&quot;mq&quot;:640,&quot;dvb&quot;:&quot;0 0 1910 1000&quot;,&quot;mvb&quot;:&quot;595 80 960 720&quot;}" data-astro-cid-t4uphou2=""><svg viewBox="0 0 1910 1000" preserveAspectRatio="xMidYMid slice" aria-hidden="true" focusable="false" data-astro-cid-t4uphou2=""><defs data-astro-cid-t4uphou2=""><pattern id="cover-veil-dots" patternUnits="userSpaceOnUse" width="40" height="40" x="-5" y="0" data-astro-cid-t4uphou2=""><circle cx="20" cy="20" r="4.2" fill="#A9B2BC" data-astro-cid-t4uphou2=""></circle></pattern><mask id="cover-veil-mask" data-astro-cid-t4uphou2=""><rect x="0" y="0" width="1910" height="1000" fill="#fff" data-astro-cid-t4uphou2=""></rect><circle data-loupe="" cx="1240" cy="500" r="232" fill="#000" data-astro-cid-t4uphou2=""></circle></mask><clipPath id="cover-truth-clip" data-astro-cid-t4uphou2=""><circle data-loupe="" cx="1240" cy="500" r="232" data-astro-cid-t4uphou2=""></circle></clipPath><filter id="cover-lens-warp" filterUnits="userSpaceOnUse" x="1008" y="268" width="464" height="464" color-interpolation-filters="sRGB" data-astro-cid-t4uphou2=""><feImage href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAIAAABMXPacAAA3j0lEQVR42u2ciVvVdfq/z7SZ01hp5VZpmY6pueVCLmCKCyqCCyoogoLsOxwOZ4X77Odw2HcQFERBBRXccEETTMMlt7IczdIWtUUrpzHb5vpd5wMom1oz3+/3N1Oc6/UX3PfzvJ734bo4on79+Gs/+vfl5Zd4uQ8D+jDwRQa9wCu9rRnciyHPM/Q5hj3H8Gd5tScjejCyB6O6M7obNl0Z05WxzzDuacY/hW0XJnTh9c5MfBL7J5j8BFMeZ1onHDox4y/MfIxZj+H0Z2Z3ZG5H5j3K/A4s7IDrIyx6BPeH8XgYz4dY9hDeD+LzIH4PEPAAQQ8Q8ifC/kTEA0Q9SPTDxDyCrAOKDqgeJe5ReBTNo+g6YOiA6RHiHybhQZIeIOVPpP2JjAfIeoCcB8h7kPwHWfkQhQ9R9DDFD7P2EUofYV0HNnSg/FE2dmRzRyr/zJbH2PYY2/9CVSd2dmL34+x5guon2Pckb3Rmfxdqu3DgKd58moPPcKgrdV053I0j3Tnag2M9eLsnx5/lxHOcfI5Tz3O6F+/0tubdFzjzIu/14f0+vP8SZ/vyt36c64eo31+5h4PBd3PQndHdsenGa10Z80yDA9unsOvChM5M7MwkwcGUx5n6uFXAdEGAoyDAuSNzBAEuj7KgUcDih1nSVMBD+HUgsCPBjxHaifAniOyMuAuSp5E+g7wriq6ouhHXDbqh6YauK/quGJ/B/DSWLiR2JvkJUjuR/hiZHcnpQN5DdwSsfpg1jQLWd6DsUauATR2pEARsFQTsEATsetwqoPoJ9j7Jvs7s70xNF2pv03+Gt7pS143D3TnavW36p+9J/9xfEfXtz30dDOnF0OcbBTzLiJ5NlqDRQcMSCA4aluDJtpbgzzjXL4EgoH4J3OoFPMLSjnh3wudJ/J8i8BmCuxPak/DniOyFuDfRLxLTB9lLyPui7IuqH3H9oB+afmj7ou+L4SVMfYh/EUtvEnuR/BypPUnvTuYzZD9F3pPkd2JVR1Y/YhVQ0jj+ZY+y8VHr+Ff8mS1/bmv8n2wY/3r69eN/h/7t8e/J2882CnieU7+C/vn+iF56mfs76NXcQc/mDpoXkV1jEd1jCWY3LoFVQEcWdcKjM0ufxrs7vs/i34vAFwh+idB+hPcncgDiQUS/QswQpEORD0MxHOVwYocT9yq8ino42uHohqMfhnEopiHEv4JlEIkDSO5Paj/SXyLzBbJ7kfss+d1Z9TRFnVnbiXUdG/pnU2P/3G3868un5h7l07PJ+NfT73V/+h+8jKjPy7TpYEAfBrzY3MHzzR00HoM7RfQ0459utgSTWi2BY+MSzOnI/Mdx7cLibnj0ZFkvlvfBtx8BLxM0iJDBhA0j/FUiRyIeTbQNMWOQjkU2Dvl4lLaobIm1Jc4O7FDborFFa4t+PIZxGMdiGkO8DZbRJI4k+VVShpE2mIxBZL1Mbj9W9GFlL4p6sqYbpV3Y8LjQP8L4b201/ntbjP/THGwy/m1U//N3qr+Bfr2AtuhfsAoYwL/jYFTzY9DiGre5BE6dmNuF+d1wexb3F/Dsi1d/fAbhN4SA4QSPJMSGsDFEjCPKDvEEoicSY490MrIpyKeimIbSAZUDsQ7ETYfpqB3QOKB1QDcN/VQMUzBOxmRP/EQsE0iwI2kcKWNIsyF9JFnDyRlC3iAK+lPYl+IXKHmWDd3Y2IXKTvca/9u3t2n1H+n+b9G/MADRiwO5t4OBrR00HuS7FVGLa3x7CWY8ifMzzOvJgt4s6suSl1n6Ct7D8B2Bvw2BYwm2JXQC4ZOInEzUVMQOSGYQ44h0FjIn5M4oZqOcg2ousULi5sE81HOt0cxFOwfdbPTOGJwwzsLkiHkGFgcSppI4meRJpE4g3ZbMsWTbkDuC/GGseoXVL7O2L+t7U96TimfY/mTz8W99e1uVT+tnTz39M/ej/+FARC8M4tc4GPRCMwdtHoO7FZH9E0ztwszuzH4elz4s7M/iQXgMZdlIltvgN44AO4ImEjKZsGlEzCDSEbET0bORzCXGBel8ZAuQu6JwQ+mGahGqxcQuJk4IQtSL0SxGswitGzo39K4YFmCcj8kF81ziZ2NxItGRpBmkTCNtMhkTybIjZxwrbFg5kqKhrBnEuv6U9WHz82zrzq4ud56ercunzepvSv9O9d+T/keDEPV+hTsOBtzHQRuPosZj0LqI6pdg0lNM64FjL2b3Zf4AXIfg/iqeo/Eai48d/pMInEKwA6EzCXcicg5R8xDPR7KQGDeki5G5I/dA7oliKcplqLysifW2Jk4IQtRCNF7WaJehW4reE70HBneMizG5YV6IZT4J80icQ7ITqTNJdyBzCtmTyLOjYCyFoyl+ldIhbBjA5r5s7cXOHux9qmH8W5dPG9Xf+tnTJv0Bd+hffEUQcNvBi/+GgzaLyL4HDr2Z1Y85A5k/FLeRLHmNpbZ4T8R3CgEOBDkS4kzYXCJciFyI2I1odyQeSD2RLUPmjXw5Ch+Uvij9UPmjCiA2gNhA4hpDY9SBqAPQBKDxR+uH1hedD/rlGLwxLMPoidmDeHcsbiQuJMmFlLmkOZPhSJYDuVPIn8gqW1a/RslINgxl00C29KOqN9U92i6ff5n+h03oWwX0Gkzvwfd00LctB80Pcusisu2BfW8c+jFrIHOHsWAUi8biYceySSyfit8MAp0InkOoC+ELiVxE1BKiPZEsI8YbqQ8yP+T+KAJQBKEMRhWCKtSa2DAh4cQ1hsaow1GHWaMJFRKCNhhdELoA9P4Y/DD6YPLGvIx4TxKWkLiI5IWkupA+h0wncmawYiorJ1Fkx9qxrB/FxmFsGUhVP6p7U9ujjfK56+G9Tb/vPekP5tJgRM8PoaWDgXcc9L2vg+cY1vzbmU137Hph35fpA3EayrxRLBzL4gl42uPlgI8j/s4EzSNkAWFuRLgT5YnYC8lyYnyR+iMLRB6EIgRFKMowVBFCIomNIlZsTVx0s9A8arGQKDSRaCKs0YahC0UXgj4IQyBGf0y+mJdj8SLBkyR3UtxIW0DGPLKdyXOkwIFCe9ZMYN1YykdROZQdA6nuS00v6ro3+8514tk2Dm8L+ueb0h/Ykv7HQxA9NxSrg3oN/6qD20Vk8xx2LzF5ANOH4jSSeWNwtcPdnqUOeDviO5sAF4IXErqYcA8ilyH2JtqXGH+kgciCkYeiCEcRgTISVRQqMapoYiXExlgTJxUiaxZaRGqNOkaIBE00GjGaKLSR6CLQhaMPxRCMMRCTP/G+WLxJXEayB6mLSV9Ilgu5s8l3ZJUDxfaU2lE+hsqR7BjKngHUvETdc3fK51+mf2kwl4ZY6X8yFNFzw7A6GNrMwe2z/Gsc3C4imxew68/kwcwYgfNruNjhas+SaSx1ZPls/FwIdCXEnTBPIryI8iHaD0kg0mBkocjDUUSiiEIpRiVBFUOslFiZEDlxcuIUQpQtQ+sohMhRy1HLhEjRxKCRoBWji0IXiT4cQyjGYMyBxPuR4EOSFymepLmT6UqOCytms8qR1dMotafMjorX2D6CPYOp6U/dC3fK59fT/2hQc/pDrfQ/GYbo2eFYHQyzOnj+X3JQf5Bt+mI7kMnDmTEa5/G4TMRtKktmsmw2Pi74uxLkTqgn4V5E+iD2RxJETAjSMOQRVu6KaJQCd5W0AXrsbdyqxsS2EdqMqjGCErWiQYZGMKGVoIu2mtBHYAzDFII5CIs/iT4ke5HqSYY72a7kubByNqtnUjKVsolUjGf7aPYMp3YgdX0bDu+/QP/jevrDrPl0OKKer/LscOo1NHXQ9Hna0sFLzRwMeZHR/bEdgv1IZozFeQIuU3CbgYcTXvPwWUjAYoI9CfUiwocof6KDkIQiDUcWiVyMQoJS4K4SoDdwb0o87l7h3mnqo96EAo3cakIbg06CXowhEmM45lDig0jwJ8mHVC/SPclaTN5CCuZR5ETJDMqmUDGB7WOpHkntEA7359SLzem/dFf69Q+epvQ/HW7NZ68i6jECq4NXWzkYfD8Hwh8qBvdl1EDGD8fehum2OE/CxQG3WXjMwWsBvosI8CDYizAfIvwRC+hjmqKXopI1olfeB/pv+NxThlrZqEGGVnpHg0nQYAkiyZ8UH9K9yPIgdxEFCyiaQ8ksyhyomMQOW6ptODCcIwM53df6x4b3+9yP/uBW9F+10r88AlGPkVgd1Guor6PGs9zawUvNHbzyV0YOZtwIJo1h+gScp+Ayg0XOeLjg5YqvO4FLCVlOuB+RgYhDkIQjjUR2G70clcA99i7c/8c+bZlQCyY0gol6DQYxxkjM4VhCSAwk2Y+05WQuJdedAleKXFjrTNkMKqawYwJ7x/DmCI4O5p2/NqP/QZv0G09uPf3PXuWzEVb6l0ci6j6K7iPpMbK5g2Etz3JrBwNfZsRQxo5i0ngcJuI0DRdH3ObgsQDvRfh5EuhNiC/hAUSFEB1GjIBeLkEhRXlP9P+Ln7tr0MrRSdELGkyRxIeREEJyAGm+ZHqT40n+IooWUDKHMkcqp1E1kb3jOTiKY0M58/Jd6d85ucOa0x/JlZFcGYWo22i6j6b7KBpW4S4nodmfjPrz8kCGD2eMDa/bMc0ep+m4OOE2Dw9XvN3xW0rgckL9iQgiKhRJBNIoZPXoZc3R/99wv5+JOxpkVg0GCcYozBEkhJIURKo/mcvJWUq+O0WulMyjzInK6ey05w07DtlwfDjvD7TSv9CKfuvSrx/8K6O4MpqroxF1taGbDd2aOmiyCq0dvDiAv77C0BHYjGHCBKZOYdZM5s3GbT4ei/D2wM+LIF9CA4gIQRyOJBJpNPIYFDJh8P+/o7+nBo1SWAUZ+hiM0ZgjsYSTFEJqABm+5HiR70HRIkrmUz6bLTPZNYX9E6gbw8kR/O2VZl90W9BvWjtXRlnRX7XhcxtEXcfQ9bU7Duo1tHESGh30HczgkYwai93rTJmKoyNz5+C6gCWL8fLEz5sgP0KDiAhFHIFEjFSC/HbnqIhtXjj/EZ+mDuo1CI2kl2KUYBZjiSAplNQgMvzI8abAk9WLKV3AxjlsdWT3VGpe58hYTo/k/OBm9FuWfv3g36b/Gp+PQfTMWJ4RHNSvQht11OSb2otDGDiKEeMYPwn7acycxZy5LFzIEne8luK7nEB/QoOJCEMcSYwYWQxyGUpFG4P/H/dpvQoK9DIMMZjEWCJJCiM1mEx/cpdTsJTV7qxbyKa5bJtF9TQOTOLYOM6M4sMhd75nfTqsrdoRBv/z1/hiDF+MRfT0OBocNF+F23XU9IXafzTDxzNmEhMdmO7E7HkscMV9CcuW4etDYAChIUSEI44iJhqZFIVcoF8/+P+x6NvSoI5Fo7I60MkxSDFFY4kiKZzUEDIDyPVh5TKKl7Delc3z2OHEPgcOTeL4eM6Obv7WbF07wuDX0/9yHKKnxvP0OO5oqF+FJpf5dh31tWGILTb2TJjONCecXJjvxuIlLPXCx5fAwCb0JY30lY21819Bv5UDtQqtstGBpImDQPJ8WeXFmiVscKPShZ1O7J9OnT2nbDlv06p2hHvbdPC/HGfNV+MRdbHlKVsaNNx9FV6wYaAtI+2xnc4UZxxdmOfGIg88vVjuS0AgIaFERCAWC/Rld+jfrp3/sk+TOmpwIBMciEmKIC2UrEBW+FLoxVoPyt3Y6sJuZ2qnc9SeM7Z8ZHPXwf9ybAP6r2y5ZouoywS62DU6GN/WKoziORv62zF8MmNnMMmZGS7MccPVAw9vvP3wF+iH19OPEejXl/5/1+Dfu46U6BSCgxirg2TBQXYg+X4UeVPqwSY3truw15mDMzg+mbN2fGLTQL/l4I9vpG/HtQmIOr9OZ8HBHQ3NV6G7DS/ZMXgyo2cwwRkHF5wXscADd2+8/PALIligHyVG8nuif3cH5hgSBAfpoeQEUeBHsTfrPahYRJUL+505PIPTk/nAjit3G3w7a65P4PrriJ6cSOeJdH6d+lWob6Smq9DLjgGTGTGD8c5McWGWGy4eLPJmqR++QQSFEtacvlL1e6HfyoFW1cxBSgQZoeQGscqPtd6UebDFjd0uHHDm2Azem8wlu+aDL3RO/eBff53rE/l6IqIn7HlyEk8KDm5ruL0KPWzpa8/Q6bzmzEQXZrgxxwNXLzz9WB5IQCihEUSKiY5BKkP++6PflgO9AqOM+BgSxaRGkBXKikAK/Sj1YpMH293Y58Jbzpycznl7Lts2H/x69K9b0X89iW/sET0xGauDphqaNFJvewZOZ6QTdi5MdcPJgwVeuPviHYh/KCERRIgR36av/D3Sb+1A2eDAEkOSmLQIskPJD6TYl/VeVHqwy40aF446cWY6F+1bdk49+q/trfS/mYzo8Sk8PoUGDZOs6dyooYc9/RwY6sSYedi7MnMJ87xY5MvSQHxDCIogXIxYQkwjfdXvlX5zB5pGByYZFgnJYjIiyA1hVSBrfSn3YtsSql05NI+TTpxz4LJ9A/rr9egnNaD/doo1ok7TeHwqLTVM5KlJ9HZg4CxGzcPOlWlLmL2MhT54BLA8hIBwQqOIlCCRIpOjEOjH/o7pN3GgFhzolBjkmKUkSkiNIiucFSEUBbDOh83L2LmEGleOzOPMLC468NWkO51zG/23U7kxDVEnB6xppaHHNPrNYthcxi7E3h3Hpbj4sDgAr2D8wgmJIkJCtBRpPf0m1f87/zT5nqxTYpQTLyVJQloUOeEUBLMmgDIfti6l2p2DCzkxl3OzuDytDfQ3HKwR/WUGf5lOCw1dptLLkYFzGLWACYtxWMrs5bj64xmMTxiBkYRFEyUlRo5c2ezw/iE+TQ+yEpOcBCkp0WRGkhdGYTCl/mxeTtVS9i/myALOzOGSI9emtkT/9+n8fQaix2by2ExaaOg+k76zGTqfsYuw98TRGxc/3IPwCsU/ghAxkTFIZMgUKFR3qv8P9Gk8BjoVBgVmGYkxpInJjqAglOIgyvzY6k21JwcXcXI+52dzZWZL9N/NtEb02Cwec6Sphidn8PxsBsxnpBt2Hjh4MdsX10A8Q/GJIEhMuASxDKnCOv5/iOq/5zHQKzEqsMhIlpAhJi+CwlBKA9nsS5UXNR4cdeO9+Xw8m69nNEP/nSPfzUL0ZyesqdcgmHjGmT4uDHHltSVMWoajDy4BuIfgFY5/FKESIqVI5Mj+gOVzlyIyKDHLSZSSKiE7ioJwikMoC2CrD3uX8dYSTrlywYUvnBu5O/KPWfzDyRpRx9l0dObPzg0aOjnRcx79F/KqO7ZLmbocZ38WBuMZhk8kQdGExzSMf335xP4x6Tc6UDcWUcMSxJARTV4khWGsC6bCn13LqV3K2+6cXchn87jh1IjemZvO3JyNqONcOs7BqmG2VUOXufRewKDFjPZkojcz/JgXxOJQvCLwFwvjL0MilI/yj1k+bRWRVigis4JEmbAEYgoiWBNKeRDb/djnzWFP3l3MxQVcmyugny1kDjfnInp0HtbUa5hDt/n0XcRQD8Z6McUXp0AWhOARjk8UQRLCpYjlSJV/1Nt7z2tsVGKRkywlQ0JeFEXhrA+hMpDdvhz04qQH5xdxdb7AfQ7fz+X7edaIOsznURch83h8Ps+68fISRi7DzgeHAOYE4xbGskj8ogmJaRx/Vfv4t7UEqoYlSIshJ5qVkZSEsSmYqgBqfDi6jPeX8Kkb384X0LtYc2s+og4L6bCADvOt6eLKC+68shSb5UzyxzEIl1DcI/AWEyghrH38f90SpEjJlJAvpjiCslC2BrHXn7rlvLOUj9y55mrlbs0Cbi1E9Igrj7jSYSEdF9JtMX09GebNOD+mBOIcwsJwPKPwjSZYSoTcOv6y9vG/+xIYhCVIkpMuJTeawijWhVMRwu5A3vTjhDfnPbm6mJsLreh/cLVG9MgirHGj0yJ6etDfixG+2AXgEMzcMBZF4iXGP4ZQGVEKYpTW/vmjP37u/hzSqzApSVCQKiM7hgIxayPZGEZVMDUBHPPlrBefeXBjET+48cMia0QPu/PwYms6e9BrGYN8sPFnYhAzQ3GJYEkUyyUESgmXE61AqkLRPv53XwJdLEYV8QqS5WRKWSFhdRRlEWwLZV8Qdf6868OlZVz34MfFQtwRPezBw0t4ZAlPL6XPcob4MTaQySE4hbMwEs9ofGMIlhGpQKJsdn7bP62XoOEUK0lUkC4jN4bCaNZFUhnOnhAOBnLKjwvL+XIpPyzhxyX86IHoIU8e8qTjUrp708+X4QHYBjMtjDmRuIlZJsFfSqhc6B9h/FXt43+PUywsgUkltJCcbCkrJZSI2RTJzjBqgzkewDlfrnhzcyk/eVojemgZDy2jkzfP+vJyACODmRDKjAjmReEejXcMgTLCFYiV7f3zG1rIoiRZQaaM/BiKoymPYnsE+0M5Gsz7AXzqyw1vflpmjehBbx705kkfevkzKAibECaF4xjJfDEeEnykBMmJUCBRIYtFGdfeP/droTgMsZhVJCnIkJMnpUjCBjFbI9kbTl0I7wZxyZ+vffjZ2xrRgz485MNT/rwQyOAQxoQxORInMQsleErxkxGiIFJp7R95e//8uhbSCy2UqCRNQY6MQinrJFSK2RPJoTBOh/BRIF/585MPP/sgesCPDn48E8hLwQwLY1wEU6OYHY1bDMtk+MsJVRClQhrbfgB+wxkwxpKgIlVBtpyVMkpi2BzNrijejOBEGB8E80Ugt/z4xQ/RAwF0DKR7MP1CeTUcuygcxMyVsFiKt4wABWFKxO3989tbyKIiRUmWgnwZa6RslFAlpiaKt8M5F8qVYG4G8ksAogeCeCyYnqH0D2dkJBPEzJDgEoO7jOVyghREqIhuF/DbBcSrSFKRoWCFnGIZZTFsl7BfzNFIzobzWSjfBfNLEKI/hdAplOfCGRDJaDETo3GMYb4UDzk+CoKVVgGS2PYD8JvPgDnWKiBdSZ6CIjkbpGyNYV80h8W8F8kn4dwI5Z8hiP4UxhPh9IpkkBibaOxjmCVlgQxPOb5KQlREqoiJRR7XLuC3CIjDFEuiijQVuUoK5ayXsUVKdQx10bwr5lIk34TzzzBED0TQOZLeYl6JZkwMk6U4yXCVs1SBn5JQFVGxVgGKdgG/6Q4LAhJiSVWRo2SVglI5lTL2SDkUwzvRXBRzPZJfIhA9GEUXMS9GMySGsVKmyHCW46ZgmRJ/lVWAOBZpnFVA+wH49WdAF4cxDosgIFvFSiUlCirk7JZxUMqpGD6M5pqYn6MQPRzN0xL6xDBUyjgZU+XMVuCmxEtFgIqw2AYB7Rf4t97hegEpsWSpKFBRomSzgl1y3pRxUsqFGL6U8GM0okdieEbKSzKGyRkvZ5qCOUoWqQQBsVYB0XHI2gX8Cw+hOOLjBAGxVgFrVWxSslPBATkn5Hwg4wspP8Qg6iCjq5y+coYrsFXioGSuisUqvGMJjCW8XcC/JyA5lsxY8mNZo2KjiioltUqOKzgv53M5t2SIOijoqqCvkuFKbFU4qJgby+JYQUAc4XHtAv49AXFkxgkCYtkYS5WKWhXHlZxX8rmCWwpEj6ropqKfiuGx2MbiECsIiMM7rl3A/5yAONbECQJiqY3leCznVFxV8b0K0aNxdIujXxyvxmEXx/Q45sW1C/hfEVAex444auJ4O45zcVyN4/s4RI9afwCeftYfwcbO+kPAzBN+DNVb+DXIcOG38GTCL0/FCs/c9s99P2rQggHiIRkyIR/WQDnsgBp4G87BVfgeRI9q6Kahn4bhamzVOKiZq24X8L8iYKOaKjW1ao6rOafhqobvNYg66Oiqo6+W4VpsNThoBAFqvNXtAv7nBKhZoxYEaKjVcFzLeS2f67ilQ9TBQFc9ffUM12GrxUHLXA2LNYIANeHqdgH/ngA1mWpBgIaNGqq01Go5ruO8ns/13DIgesTEM0ZeMjBMz3g903TM0bJIg5eGADVh7QL+PQEparLUFGhYq2GTlp06Dug5oecDA18Y+cGE6OF4njbTx8RQI+MMTNUzW4ebVhCgsQoQq60/htou4LcKMIJFLQjQWAWUaNmsY5eeNw2cNHLBxJdmfoxH9GACXSy8GM8QE2ONTDHgrMdNxzIt/hpCNQ0CFO0CfosAXaOAVA3ZGlZqKdFRoWe3gYNGTpn4MJ5rFn5OQPRAEp0T6W3hlXjGmJhsxMmAq56lOvy0VgFRamLUVgGqxn+Vbf/c+6MRBJjUJAgCcrSs0lGqp9LAHiOHTLwTz0UL1xP5JQnRn1J4IpleiQyyYBOPvYlZRhYY8NTjqyVEQ6TGKkDeLuC3CNALAhI1pGnI1VKoZ72BLUaqTdTF866FS4l8k8w/UxD9KY1OqTyXzIBERluYGI+jiflGPPT46AjWEqFBokaubhfwWwSoMatJ0pCuJU9HkZ4NRraa2BfPYQvvJfJJMjdS+WcaogcyeCydnqn0T2ZkIhMszDDjYsLdwHI9QTqrgGgNMnX7Hf4tTyA18RqrgAwdK/QUGygzsd3MfgtHEzmbzGepfJfOLxmIHsiiYybd0+mXyqvJ2CXgYGGumcVGvA0E6AjTIm4X8NsFWDSkaMnSkW9gjZGNZqos1CTwdjLnUrmSzs1MfslC9EAOHXJ4JpOX0hmWwrgkpiYwOx43E8sM+OsJ1RGlQapG0d5Cv/ICqzGqSdCQqiNbz0oDJSY2x7MrgTeTOJHCB+l8kcmtHH7JQfRgHg/l8VQ2L2QyOI0xKUxOxMnCQjOeRvwMhOiI1BKjaT8Dv+EAmDQkaknTkWOg0Mg6M5UW9iRyKIXTaXyUyVfZ/JTHz3mIHsznwXyezKNXNoMysEljUjKOicy34GHGx0iQnggdkvYW+i39Y9aQpCNDT56RIjMbLGxNZG8ydWm8m8GlbL7O4+d8a0QPreShlXTK59lcXs5iZDoTUpmRxLwE3OPxNhFoIFyHWItUY22h2PYluPdXMDVGDRYtyToyDeSbKI6nPIHtSexP5Wg672fxaS438vlppTWihwp5qJCOq+ieT79chmdhm860FOYk4mZhmRl/I6F6onTWFmo/A7/mAJg0JOhI1ZNtZKWZEgubEtmZQm06x7M4l8uVfG6u4qdCa0QPF/Hwah5ZzdOr6LOCITmMzWRyGk7JLEzEMx5fE8EGInVItMg1KNXtLXT3/lGj12DWkqgj3UCuicJ41iVSmcyeNA5mciqHCyv4chU/rObH1fxYhOjhYh5eY03nInqtZFAeNtlMzGBmKi5JLElguZlAI+F6onXtLfSr+ideR7KeTCMrzKxOoCyJbansy6Aum3fzuLSS60X8uEZIMaJH1mJNCZ3W0rOI/gWMyMUuC4d05qawKBEvC/4mQg1CCwlLoGpfgrboa4TxN2mF/jGQbaLAwtpENqZQlU5NFsdyOVvAZ0XcWMsPJfyw1hrRI6U8UkqHdXRcR7c19C1kWD7jcpiSiXMaC5PxTMA3nmAjEXokOmRNWqjdQYv+MWgw60jSk24kN57CBNYlU5HG7kzezOFEPucLubqGm+u4tY4fSq0RdVhHh/V02GBNl1JeKOaVVdisYFI2jhm4pOKehLeFQDNhRsR6pFoUwhK0C2h2ftXoNBi1WPSkGMk0k2+hOImyVLZmsDebuhW8s4qPirlWyq0NQtZbNYg6bODRMiHlPL6BZ0t4eTUjV2KXh0MWc9JxS2FZIn7xhJiINFiXQN6+BHc7vzoSDaSZyIlnZSIlKWxKpyqLmjyOruT91Xxawrcb+L6c78usubUB0aPlVvSPbqTjJmu6baDvWoYWMbaAKbk4ZbIgDY9kfBIIMhPevgT3G/9kIxlm8hIoSmZ9GpWZ7M7lYAEnizi/lqsbuLnJmu83ChrKEXWsR7/Zmj9X0GUjvdczaA2jC5mYz4wc5mWwOBWvJPwthJobl0DbvgTNx1/bMP6pZrItFCSxJpXyDLbnsC+fw4W8u4aL67m2kX9UcHOzkE3c3IjIir7Civ7Plfx5C50q6VlO/3W8WoztKqauwDmbhel4puCTSFA84SbEBqS6hiX4gz+H1LfHX4fFQLKJjHjyEilMYV06FdnsWkHtKt4u5uw6PivnRiX/2MI/KgUNggmRlbuA/rGtQrbxTAV9yhhSymurmbQSxzxcsnBPwysZ/wRhCYxI9Mi0KDXE/rGLSK1Gq8Ggxawn0SiMfwIFyRSnUZbF1jz2ruSt1Zwq5UIZX1Tw3Ta+22pNg4ZKRI9taeD+2Db+sp2/7ODJ7Ty/mQEbGFmCXREOBczOxTUTz1R8kgiyEG5uWAK59s4SxP1Rx1+vbRx/MxkW8pIoTKU0k825VBVQU8TREt7bwMeb+Xo7f9/B37cLGupNbEHUFH2nKiE76b6NvpsZuoGxa7EvxDEflxzcM/BKxT+JEAuRJiQGZLo/9DWuLx+DDrOBRBNpFrKTKEilOIOyHLbmU13IwbWc3MD5zVzZxo2d3KiypqkGUQv0j+/i8d102UWvrQzcxKj1TFiDwypmr8A1G890fFIITCQsnigjMXrhGv8hi6i+fPRaTHoSjKTEk5lIXgqF6ZRms3kFVavYv4Yj6zmziUtbubaLb3fz7a6WGkQt0D++myf28EQ1PXbSbwvDNjJ2HfbFOK7CJY/FWXil45dMSAIRZqKNSPUotKj+YA7UajQadFqMeuKNJJlJSyAnmYJ01mRRlsfWVVQXc3AdJzZybguXd/JNNd/sERw01yBqjf7JvTy5j6f20ruKgVsYVY5dKdNWM3slC/PwyGJ5GgHJhCYQaUZiRFbv4A9zDBpePloMesxGEs2kJpCVzIo0irJYl8fmlexcTU0pR8o5s4WLVXy1l6/38fXeNjSIWqLfS+d9dH6DzvvpUU2/KoZWMqYc+1JmrmZeAYtyWZqJbxpBSYRbEJuJMSDXodTeOQZxv/fq12rR6zAZsJhJtpCRRG4aqzJZm0t5AdtWU13KoXJOVnKuisvVXN/P9Te4LjhooUHUgL5x8OvRd6mx5qlaelczcAcjK7ErY2oJTkUsKMA9F+9M/FMJSSLCgtiEtNFB7O/dgbqRvtGAxUSShbQkslPJz6Q4l/UFVBaxq4SaMo5WcmYHF6v5qpZrNdbUa6hfha+rGzSInmiK/g267G9A/9QBnn6THrX0rWboDl6rYGIZM0qYU4RrAZ45LM8kIJXQJCItRN92oPk9O2g4vAL9eBOJFlKTyEplRSaFOZQWsKmI7SXsK+OtCk7u4Hw1l2v58k2+OtCooX4V3rjTSCIr+n130HeptdJ/WqD/zEGeOUSvGgbsYcR2xlcwpYxZJbgUsSifpTn4ZhCUSlgSURYkJmQGFL9fB/X0dToMBswmEiykJJGRSm4Gq3JYm09ZEVtK2F3GgQqObee9PVyq4YtDfHHQ6uDLegdNNeyzahC16Jz6wX9aQN/1EF3fonsdL9UweA+jtzOhAocynNeyoAj3fLxy8MsgOJXw5g5Uv7susj57tM3oJyeRnkpOBgU5FOezvoiKtVSVsb+Cw9s5vYcParhSx+dv8fkhq4YvDzZfhcZGEjXrnCaD3/UtutbR7TDdj/BcHf1rGL6HsduZVMGMMuaU4FqERz7eOfhnEiI4EFuI+T06aErfZMIi0E9LJTuT/ByK8iktYlMJ28vYW8HB7Rzfw9kaPqnjyhGuHuZzQUOzVWjSSKL6zmk9+N0a6fc4So9jvFDHwFpGVmO7gykVOJYxr4RFRXgWsDyXAMFBRL0Ds+BAL9xkDXH/1d8P1Kg11qur0wv0zVb6SQL9rExW5FJYwNoiykvYWsbuCmp3cLSaM7V8VMflY1w+2uDg6t1WoRZRA/rWg99Iv+cxer7Ns8fpW8eQWmyqmbCDaZU4lTG/hMWrWVqATy6BmYSmEZGMOEFwYLzj4L90FdRN6RsF+gkkJZOaRmYmebmsKmDNajaUUFnGzkr276CumlO1nK/j0+N89jafNTq4critVRA0iO6gb2vwex7jWYH+cyd4/iT9DzP8AGP2MrGK6ZXMLmdBKe6rWbYS3zwCs5o4iG90oGv2PTnuv6h2NGh1jfTjm9DPIjePlSspXs36UjaXs6OSfVUc2svxA5w9zMcn+eSE1cGn9Q7usgr1GkTNBr/uzuD3aBz8Z0/w3Ekr/V6nePEUA48w4k3G78V+JzO3MGcjC9expBivVfiuIDCb0HQiUhAnEmNBZkJuEBw0r6O4//ja0Wit9PUGDCZMFiyJJKWQmk5mNrkrKFjF6mLWrWPTRrZtoXonB/Zy7E3OHOHDU1w6JTg4yacnGlbhctNVqGu2CqIWg9+idp6rp3+KXqfp/Q4vvEvf0ww+yqiD2L3BlF04bmXuJlzXs2QNXoX45ROUQ2gGEamIk5BYkJqRG1HqUeka6+g/VoO6oXY0OrR69EaMZswWLEkkpZKaQUYOOfkUFLJ6DaXr2biJrVvZvYuaNzhykNNHOX+aj97l4jtcOs3Hp6wOPjnRqo6ar4KofvBb1E7PJrXTlP6LZ3jxPf76DkOPYXOICfuZuptZ25i3GbcNeKzFuwi/AoJyCc0iIg1xMpJEpPHITSgMgoZWqxD3nzb4enQG9CaM8ZgTsSSTlEZqFhm55BSQX0TRWko2UL6ZLdvYtZv9+6k7xMlj/O0dPnyPD880c/BxCwct6qgOUdN7e6d27kK/z3v0eZ+XzvLyGYYfZ0wdr9cwrRqnHbhU4laORynexfitInAFodlEZBCViiQJaQIyMwojSoOwCv9RGgT0aq0w+AZ0RgxmjAmYk0hIJSmD1GwyV5CzivxiikopKaesksod7KzmjRoO1XH8OO+f4YOzXHifC3d30EYdHUZkrZ2jrWrnREPpt6D/kkC/79/od46B7zPiJGOPMOkADvtw2onLVtw24bEe77X4FRKYT0gu4VlEpRGdQkwiMgvyeg365hr+v5hQN0evt6LXmzFYMCUSn0JCGslZpOWSmU9OIflrKVpPySbKtlK5k6p97D3AwSMcO8mZ9zl3jvN/szr4oJWDS40OWtaRoEHUunYaSv8Uve9O/6/n6f8Br/yNkacZd4xJh5i+H+fduGxnUQUeZXiV4ltM4CpCVhCeQ2Qm4jQkyUgFDQozSiOqe2qI+79Cr9GjNaIT0BsTMSdjSSMxk+Qc0laQuYrcYgpKKSpjbQVl26nYzY797D3Em8c4epp3/sbZD/jb+bs6uFjvoPVJEDSIGmqndem3ot+3Of2XLzDgQwafZ9QZxh/Hvo7ptTjvxaUKty14bMJrPb5rCSgiuICwPCKyEWcgSSUmGVki8tsaDFYTsYKJuLuY+J/xoa7/W76Vu1rgrjE0oNdbMCRiSsaciiWDpGxS8kgvIKuI3LUUrKdoEyVbKKuiYi87aqmu48Bxjpzh9Hne+5D3LzRzcL5NBy3q6O2GOhLdrfTvQ/8DBlxg4IcM+oghHzL6LLansD/KjIM478dlN27b8ajEqxyfdQSsIbiQ0AIi8ojKJlrQIG2qwYTK2KhBR1y9iXvK+LVWGqFbI3BvQG9Ea7qD3iigj88gIZukPFILSC8kaw156ygop6iSku2U7aZiP9sPUn2U2lMcPsupD3n3I858yHsXeP+D+zm4y0kQNS39pif3rvTPN6P/ykWGXGLox9icx/YMk48z4zDOB3DZh9sulmxj2WZ8yvAvJaiY0ELCC4jMQ5yNJIOYNKQpyJNQJKCIR2lGJZiINRCrbzRxW8ZtH81Dm9Hcgd7AXY/aYOWuMaE1o4tHl4A+CWMKpjTMGViyScwjuYDUQjKKyS4lr4yVm1m9jZJdlO2j4gDbD7PnOLVnqDvPyY85dYl3LjZ3cP6uDlqf5XoNoueafM/6F+gPFugP+4Thn2LzEXZnmXyaGcdwfguXGlyrWbKTpVtZvhm/MgJLCSkmrJCIAqLyiM5Bkok0HVkq8mQUiVYTSgsqwURsvQlBRpyeOJ0QbcvQOjoh+gbo1gjcNWa0Fit3XSL6ZAypGNMxZxKfQ0IeSQWkFJJWTGYpOWWs2MyqrazeSWk1ZTVUvMX2Y+w5Tc1Z6j7i+Kec+MTq4PS/5OD2N7VPTiC6fXKbPfZ/Nf0hHzNUoP/qZ4y4jM0n2H3A5PeYfhKno8w7hGsN7tUsrcJ7K76bCSgjeB2hawgvInIl4nyic4nJRpqJLB15KopkFEkoE1EloLKgiifWTKzJmjijEEOz0CJGa9QmIWY08WgsaBLQJqJLQpeMPhVDOsZMTNnE52LJJ3ElyUWkriF9HVll5G4mfyurqiiuprSG8kNUHmXHSfa8R80H1H3Cscu8/ZnVwclPOPVbHLR8GglnWdT65P5W+sM+5dVPGfEZIy8z6go2V7C7hP15pp/B6STzjrDwIIv341mNVxU+W/GvIKickPWElRBRTFQh4gIkK4jJRZqNLBN5Boo0FKkoU1AlCUkkNoFYizVx8c1C86gtQhLQJKJJskabgi4VXRr6DAyZGLMx5WJegaWAhEKSikkpIW09GeVkV5C3lYIqCqtZs591Byk/QuVJdpyh+jw1l6i7wpErHL3Msc94+1NO/BsObp9lUUv6792h3+++9BvL5zb90Vd57XPGfIHtZewv4nCOWWeYe4IFR1h0EI8alu1l+S78thNYSfAmQssIX0fkWqJWE12IZCUx+UjzkOUgz0aRhSIDZTqqNFSp1sSmCEkmrjE0Rp2MOsUaTaqQNLTp6DLQZaHPxpCDMQ9TPuaVxBeSsJrEtSSvI7WM9E1kVpKznRW7WLmXohrWHmT9ETaeYMsZqs5RfZHayxz6grc+5/DVOw5uF9G9HZxr6qDV00jU9MHTBv1zbdFvPLzDmpRPPX2bzxnzOWO/YNyX2H6F/WUcLjLrHHPOMP8kbkdZ8hZLa/Heh+9uAqoI2kpIBWEbiSgjch3iEqKLkRQhLUS2Elk+8hUo8lDmosxBlY0qi9gsYjOJawyNUWeizkKThSYbbQ7aXHR56FdgyMewEmMh5iLii7GUkLiOpDJSNpJWQcZWsqrI3U3+PlbVsvotSo6y4SSbzrDlHFUXqb5M7Ve8+SUHv+DQ59Q1OrhdRA0OLnH6YlsOzt3TwTuCgDt/ZvhX6Y+8zOgr2DSO/7gvGf8VdteYcJ1JXzHtMo6XmH2e+e/hegr3t/E8jNdBfGrw30vgboKrCN1GeCWRm4gqR7wByTpiSpCuQVaMvAh5IYpVKFeiKrAmNt+aOCEIUQvRFFijXYluFfpC9EUYijGuwVSCeR2WDSSUk7iJ5EpSt5FeReZusveSV0PBQQoPU/w2pafY8B6bz7P1Ejsvs/cr9l+n5hoHBAf1S1B3lcP1S/CvOmj6PBU1+yPPPek3LZ879FuVz9gvGS+Mv901Xr/OxK+x/4ap15h5hdkf43KBhWdZ/C4eJ1l2lOV1+L1JQA1B+wjZQ9hOIrYTuRVxJdGbkWwkpgzpBmTrkZeiKEFZgmotqjXEriFOCELUa9CsQbMWbQm6EvSlGNZj3ICpDPNG4jdjqSRxK0nbSdlJ2h4y9pFVQ86brKhj5VGKTrLmXdadpewCmz9m2xV2XaP6G/Z9zRuCgNqvOPAlB79so4huO2haRPdx0ORpJPo19Ns4vK2qv0X51I//xOtM+prJ3zDlW6bdYMbXOH/BvM9YcJFF51nyPkvfwfsEvsfwryPwIMG1hO4nfC+Re4jahbgKyXZitiLdgqwSeQWKzSg3odpIrJC4cuvPoao3WqPZiHYTus3oKzBUYtyCaSvm7ViqSNhF4h6S95K6n/RaMg+SXUfuMfJPsOodVr/P2vOsv0j5Z1R8wfav2XmD3d+y5xv2fs2+6w1L0GYRNTsGrQ7yr3Egujf9AR/e9fDeqf5W5WN7jQnC+E8Sxn/Kt0z9FocbTP87M7/D6QZzrzH/Km6f4v4RnufxOovPu/idIuA4wUcJqSPsEBFvElWDeD/R+4ipRroH2W7ku1DsRFmFqorYKuJ2WH+MVl2FpgptFbqd6Hdh2I1xD6Zq4vdh2U9CDUlvknKItDrSj5J1nJxT5L1LwVkKz1P8ESWfsuEqG69ReYNt37Hj71TdYNe3VgfVgoM3rrP/GrVtFVEbx6DFQb6fA9G/Q7/h3Xn38mk6/g43mPF3HL9j1j9w/gdzbjL/W1yvsfgqHp+x7BLLL+B7joD3CXqXkNOEnSD8bSKPIj5MdB0xh5AeRPYm8gMoa1HVEltLXI31p4DVtWhq0daiP4DhTYwHMR0ivg7LYRKPkvw2KSdIO03Gu2S9T+45Vlxg5SWKPmPNVUqvseFbNt2k4h9s+Qdbv2O7IKDZEtyliOpuL8G/4UDUJn3rH9paPDpbHN67lE/D7b37+Dt+h9M/mH3TKmDe98y/xcKbLLqBx3WWfon3FXw/xf8SgR8R/AGh5wg/S+R7iN8l+h1iTiE9ifwEiuMojxN7nLi3rT/ErD6O9ji64+hPYDyJ6RTx72B5l8T3SD5L6jnSPyDzI7Ivkfsp+VdY9SVF11l7g3U32XCL8u+tAjbfpFIQcI8luH2N2yyilge5xcP0Lg5E96ff4tnTovpblE/T29tq/Gc2jv/sm8z9Hpd6Abdw+4HFP7LkB5bexPsGPl/j/xWBXxB8hdDPCP+EyEuILxL9ITEXkH2A/DzK86jOEXfO+jPYmnNoz6M/j+EDTBeI/xDLRRIvkfwJqZ+RfoXML8j+iryvyb/Bqpus/oE1P1LyA+tuWQWUfc/G760C6pdgW6slqG6xBC2K6Gqzb2f3ehS15UB0X/otDm9D+TSv/jZv793G37lx/F2+Z8EtXH9gUb2AH/H8iWU/4f0zPj/hd4vAmwR/R+gNwr8h8jria0i+RPoF8s9RfI7qKnFXrT9CrrmK7nP0n2P8AvOXWK6ReJ3kb0i9Qfp3ZN4k5xZ5P5H/Myt/ovAnVv9oFbD2B0p/YL0goH4JKu65BG1c46av0hbHoPEg39eB6B70X7kb/buUz+3bW//0bGP8G/tn7u3+aRTg/iMeTQX8jN8vBPxC0C+E/JOwfxLxC1E/E/0jMT8gu4XiFqrvifve+hPwmu/R3cJwC9MPxP9Iws8k/ULKP0n7Jxm/kPULOb+Q9/MdAUU/UtwooH4Jyr9nY2MLbfmurSVofJI2vcZtFFFbDt65p4P/B/Am4dJ/vvvdAAAAAElFTkSuQmCC" x="1008" y="268" width="464" height="464" preserveAspectRatio="none" result="map" data-astro-cid-t4uphou2=""></feImage><feDisplacementMap in="SourceGraphic" in2="map" scale="116" xChannelSelector="R" yChannelSelector="G" data-astro-cid-t4uphou2=""></feDisplacementMap></filter></defs><rect x="0" y="0" width="1910" height="1000" fill="url(#cover-veil-dots)" mask="url(#cover-veil-mask)" data-astro-cid-t4uphou2=""></rect><g clip-path="url(#cover-truth-clip)" data-astro-cid-t4uphou2=""><g filter="url(#cover-lens-warp)" data-astro-cid-t4uphou2=""><g data-loupe-zoom="" transform="translate(-1240.0 -500.0) scale(2)" data-astro-cid-t4uphou2=""><g fill="none" stroke="#3F464E" stroke-width="3.19" stroke-linecap="round" data-astro-cid-t4uphou2=""><path d="M 411.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1211.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 20 h 7" data-astro-cid-t4uphou2=""></path><path d="M 131.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 411.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 571.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 811.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 931.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1331.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1411.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 60 h 7" data-astro-cid-t4uphou2=""></path><path d="M 211.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 571.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 931.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1411.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1571.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 100 h 7" data-astro-cid-t4uphou2=""></path><path d="M 131.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 891.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1731.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 140 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 411.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 611.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 891.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1731.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1891.5 180 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 211.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 931.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1051.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1091.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1211.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 220 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 211.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1331.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1691.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1891.5 260 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 131.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 371.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 571.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1091.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 300 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 131.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 211.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 571.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 611.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 891.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1411.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1491.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1571.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 340 h 7" data-astro-cid-t4uphou2=""></path><path d="M 371.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 811.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1011.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1051.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1091.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1491.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1691.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1731.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 380 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 131.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 611.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1011.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1211.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1691.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 420 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 131.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 371.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1051.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1491.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 460 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 611.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 811.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1011.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1571.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1731.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1891.5 500 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 411.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 811.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1011.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1051.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1411.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1731.5 540 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 611.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1491.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 580 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 931.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1331.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1411.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1491.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 620 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1091.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 660 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 891.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 931.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1011.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1091.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1211.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 700 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 411.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 571.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 691.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1011.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1291.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 740 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 651.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 811.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1331.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1891.5 780 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 971.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1451.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1691.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 820 h 7" data-astro-cid-t4uphou2=""></path><path d="M 11.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 531.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1051.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1171.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1331.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1691.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1771.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1891.5 860 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 211.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 611.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 811.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1371.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1411.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1651.5 900 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 171.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 291.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 371.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 411.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 731.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 771.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 891.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1131.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1531.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1571.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1851.5 940 h 7" data-astro-cid-t4uphou2=""></path><path d="M 51.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 91.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 251.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 331.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 451.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 491.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 851.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 891.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1051.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1091.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1251.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1571.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1611.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1731.5 980 h 7" data-astro-cid-t4uphou2=""></path><path d="M 1811.5 980 h 7" data-astro-cid-t4uphou2=""></path></g><g fill="none" stroke="#3F464E" stroke-width="2.7" stroke-linecap="round" stroke-linejoin="round" data-astro-cid-t4uphou2=""><path d="M 8.8 20.4 l 3.84 4.22 L 21.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 20.4 l 3.84 4.22 L 61.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 20.4 l 3.84 4.22 L 101.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 20.4 l 3.84 4.22 L 181.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 20.4 l 3.84 4.22 L 221.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 20.4 l 3.84 4.22 L 261.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 20.4 l 3.84 4.22 L 501.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 20.4 l 3.84 4.22 L 581.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 20.4 l 3.84 4.22 L 621.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 20.4 l 3.84 4.22 L 701.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 20.4 l 3.84 4.22 L 1021.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 20.4 l 3.84 4.22 L 1061.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 20.4 l 3.84 4.22 L 1101.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 20.4 l 3.84 4.22 L 1181.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 20.4 l 3.84 4.22 L 1301.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 20.4 l 3.84 4.22 L 1341.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 20.4 l 3.84 4.22 L 1501.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 20.4 l 3.84 4.22 L 1741.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 20.4 l 3.84 4.22 L 1901.2 15.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 60.4 l 3.84 4.22 L 21.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 60.4 l 3.84 4.22 L 61.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 60.4 l 3.84 4.22 L 221.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 60.4 l 3.84 4.22 L 261.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 60.4 l 3.84 4.22 L 341.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 60.4 l 3.84 4.22 L 461.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 60.4 l 3.84 4.22 L 621.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 60.4 l 3.84 4.22 L 701.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 60.4 l 3.84 4.22 L 741.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 60.4 l 3.84 4.22 L 781.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 60.4 l 3.84 4.22 L 981.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 60.4 l 3.84 4.22 L 1021.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 60.4 l 3.84 4.22 L 1061.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 60.4 l 3.84 4.22 L 1181.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 60.4 l 3.84 4.22 L 1301.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 60.4 l 3.84 4.22 L 1381.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 60.4 l 3.84 4.22 L 1701.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 60.4 l 3.84 4.22 L 1741.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 60.4 l 3.84 4.22 L 1781.2 55.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 100.4 l 3.84 4.22 L 61.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 100.4 l 3.84 4.22 L 101.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 100.4 l 3.84 4.22 L 181.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 100.4 l 3.84 4.22 L 301.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 100.4 l 3.84 4.22 L 381.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 100.4 l 3.84 4.22 L 501.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 100.4 l 3.84 4.22 L 621.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 100.4 l 3.84 4.22 L 741.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 100.4 l 3.84 4.22 L 781.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 100.4 l 3.84 4.22 L 981.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 100.4 l 3.84 4.22 L 1021.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 100.4 l 3.84 4.22 L 1061.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 100.4 l 3.84 4.22 L 1181.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 100.4 l 3.84 4.22 L 1301.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 100.4 l 3.84 4.22 L 1341.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 100.4 l 3.84 4.22 L 1661.2 95.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 140.4 l 3.84 4.22 L 101.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 140.4 l 3.84 4.22 L 221.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 140.4 l 3.84 4.22 L 301.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 140.4 l 3.84 4.22 L 421.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 140.4 l 3.84 4.22 L 581.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 140.4 l 3.84 4.22 L 941.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 140.4 l 3.84 4.22 L 1061.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 140.4 l 3.84 4.22 L 1101.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 140.4 l 3.84 4.22 L 1141.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 140.4 l 3.84 4.22 L 1341.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 140.4 l 3.84 4.22 L 1381.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 140.4 l 3.84 4.22 L 1461.2 135.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 180.4 l 3.84 4.22 L 101.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 180.4 l 3.84 4.22 L 261.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 180.4 l 3.84 4.22 L 381.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 180.4 l 3.84 4.22 L 581.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 180.4 l 3.84 4.22 L 661.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 180.4 l 3.84 4.22 L 741.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 180.4 l 3.84 4.22 L 781.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 180.4 l 3.84 4.22 L 821.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 180.4 l 3.84 4.22 L 1021.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 180.4 l 3.84 4.22 L 1061.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 180.4 l 3.84 4.22 L 1141.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 180.4 l 3.84 4.22 L 1181.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 180.4 l 3.84 4.22 L 1341.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 180.4 l 3.84 4.22 L 1381.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 180.4 l 3.84 4.22 L 1501.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 180.4 l 3.84 4.22 L 1661.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 180.4 l 3.84 4.22 L 1701.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 180.4 l 3.84 4.22 L 1821.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 180.4 l 3.84 4.22 L 1861.2 175.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 220.4 l 3.84 4.22 L 21.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 220.4 l 3.84 4.22 L 61.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 220.4 l 3.84 4.22 L 181.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 220.4 l 3.84 4.22 L 381.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 220.4 l 3.84 4.22 L 421.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 220.4 l 3.84 4.22 L 461.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 220.4 l 3.84 4.22 L 581.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 220.4 l 3.84 4.22 L 781.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 220.4 l 3.84 4.22 L 821.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 848.8 220.4 l 3.84 4.22 L 861.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 220.4 l 3.84 4.22 L 901.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 220.4 l 3.84 4.22 L 1341.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 220.4 l 3.84 4.22 L 1501.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 220.4 l 3.84 4.22 L 1661.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 220.4 l 3.84 4.22 L 1861.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 220.4 l 3.84 4.22 L 1901.2 215.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 260.4 l 3.84 4.22 L 61.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 260.4 l 3.84 4.22 L 101.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 260.4 l 3.84 4.22 L 261.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 260.4 l 3.84 4.22 L 301.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 260.4 l 3.84 4.22 L 341.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 260.4 l 3.84 4.22 L 421.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 260.4 l 3.84 4.22 L 461.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 260.4 l 3.84 4.22 L 501.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 528.8 260.4 l 3.84 4.22 L 541.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 260.4 l 3.84 4.22 L 581.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 260.4 l 3.84 4.22 L 661.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 260.4 l 3.84 4.22 L 741.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 260.4 l 3.84 4.22 L 781.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 260.4 l 3.84 4.22 L 821.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 260.4 l 3.84 4.22 L 941.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 260.4 l 3.84 4.22 L 1101.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 260.4 l 3.84 4.22 L 1141.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 260.4 l 3.84 4.22 L 1221.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 260.4 l 3.84 4.22 L 1421.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 260.4 l 3.84 4.22 L 1461.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 260.4 l 3.84 4.22 L 1501.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1528.8 260.4 l 3.84 4.22 L 1541.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 260.4 l 3.84 4.22 L 1781.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 260.4 l 3.84 4.22 L 1821.2 255.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 300.4 l 3.84 4.22 L 501.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 300.4 l 3.84 4.22 L 621.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 300.4 l 3.84 4.22 L 661.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 300.4 l 3.84 4.22 L 941.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 300.4 l 3.84 4.22 L 981.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 300.4 l 3.84 4.22 L 1021.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 300.4 l 3.84 4.22 L 1061.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 300.4 l 3.84 4.22 L 1181.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 300.4 l 3.84 4.22 L 1221.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 300.4 l 3.84 4.22 L 1261.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 300.4 l 3.84 4.22 L 1581.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 300.4 l 3.84 4.22 L 1621.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 300.4 l 3.84 4.22 L 1661.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 300.4 l 3.84 4.22 L 1701.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 300.4 l 3.84 4.22 L 1741.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 300.4 l 3.84 4.22 L 1821.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 300.4 l 3.84 4.22 L 1901.2 295.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 340.4 l 3.84 4.22 L 61.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 340.4 l 3.84 4.22 L 261.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 340.4 l 3.84 4.22 L 301.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 340.4 l 3.84 4.22 L 341.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 340.4 l 3.84 4.22 L 381.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 340.4 l 3.84 4.22 L 741.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 340.4 l 3.84 4.22 L 781.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 340.4 l 3.84 4.22 L 821.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 340.4 l 3.84 4.22 L 1021.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 340.4 l 3.84 4.22 L 1061.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 340.4 l 3.84 4.22 L 1101.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 340.4 l 3.84 4.22 L 1221.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 340.4 l 3.84 4.22 L 1261.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 340.4 l 3.84 4.22 L 1461.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 340.4 l 3.84 4.22 L 1621.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 340.4 l 3.84 4.22 L 1741.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 340.4 l 3.84 4.22 L 1821.2 335.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 380.4 l 3.84 4.22 L 21.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 380.4 l 3.84 4.22 L 141.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 380.4 l 3.84 4.22 L 301.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 380.4 l 3.84 4.22 L 341.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 380.4 l 3.84 4.22 L 461.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 380.4 l 3.84 4.22 L 581.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 380.4 l 3.84 4.22 L 621.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 380.4 l 3.84 4.22 L 661.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 380.4 l 3.84 4.22 L 701.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 380.4 l 3.84 4.22 L 901.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 380.4 l 3.84 4.22 L 941.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 380.4 l 3.84 4.22 L 1341.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 380.4 l 3.84 4.22 L 1381.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 380.4 l 3.84 4.22 L 1421.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 380.4 l 3.84 4.22 L 1621.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 380.4 l 3.84 4.22 L 1661.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 380.4 l 3.84 4.22 L 1901.2 375.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 420.4 l 3.84 4.22 L 61.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 420.4 l 3.84 4.22 L 181.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 420.4 l 3.84 4.22 L 341.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 420.4 l 3.84 4.22 L 461.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 420.4 l 3.84 4.22 L 661.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 420.4 l 3.84 4.22 L 701.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 420.4 l 3.84 4.22 L 781.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 420.4 l 3.84 4.22 L 821.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 420.4 l 3.84 4.22 L 901.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 420.4 l 3.84 4.22 L 941.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 420.4 l 3.84 4.22 L 981.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 420.4 l 3.84 4.22 L 1061.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 420.4 l 3.84 4.22 L 1141.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 420.4 l 3.84 4.22 L 1181.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 420.4 l 3.84 4.22 L 1301.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 420.4 l 3.84 4.22 L 1341.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 420.4 l 3.84 4.22 L 1381.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 420.4 l 3.84 4.22 L 1581.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 420.4 l 3.84 4.22 L 1621.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 420.4 l 3.84 4.22 L 1781.2 415.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 460.4 l 3.84 4.22 L 21.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 460.4 l 3.84 4.22 L 61.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 460.4 l 3.84 4.22 L 261.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 460.4 l 3.84 4.22 L 421.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 460.4 l 3.84 4.22 L 581.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 460.4 l 3.84 4.22 L 781.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 460.4 l 3.84 4.22 L 821.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 460.4 l 3.84 4.22 L 901.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 460.4 l 3.84 4.22 L 941.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 460.4 l 3.84 4.22 L 1101.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 460.4 l 3.84 4.22 L 1221.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 460.4 l 3.84 4.22 L 1581.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 460.4 l 3.84 4.22 L 1621.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 460.4 l 3.84 4.22 L 1741.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 460.4 l 3.84 4.22 L 1781.2 455.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 500.4 l 3.84 4.22 L 21.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 500.4 l 3.84 4.22 L 61.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 500.4 l 3.84 4.22 L 101.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 500.4 l 3.84 4.22 L 141.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 500.4 l 3.84 4.22 L 181.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 500.4 l 3.84 4.22 L 221.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 500.4 l 3.84 4.22 L 301.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 500.4 l 3.84 4.22 L 341.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 500.4 l 3.84 4.22 L 381.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 500.4 l 3.84 4.22 L 421.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 500.4 l 3.84 4.22 L 781.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 500.4 l 3.84 4.22 L 981.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 500.4 l 3.84 4.22 L 1221.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 500.4 l 3.84 4.22 L 1261.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 500.4 l 3.84 4.22 L 1341.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 500.4 l 3.84 4.22 L 1381.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 500.4 l 3.84 4.22 L 1421.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 500.4 l 3.84 4.22 L 1461.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 500.4 l 3.84 4.22 L 1501.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1528.8 500.4 l 3.84 4.22 L 1541.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 500.4 l 3.84 4.22 L 1661.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 500.4 l 3.84 4.22 L 1701.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 500.4 l 3.84 4.22 L 1861.2 495.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 540.4 l 3.84 4.22 L 21.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 540.4 l 3.84 4.22 L 101.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 540.4 l 3.84 4.22 L 261.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 540.4 l 3.84 4.22 L 301.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 540.4 l 3.84 4.22 L 381.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 540.4 l 3.84 4.22 L 461.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 540.4 l 3.84 4.22 L 581.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 540.4 l 3.84 4.22 L 941.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 540.4 l 3.84 4.22 L 1261.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 540.4 l 3.84 4.22 L 1581.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 540.4 l 3.84 4.22 L 1821.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 540.4 l 3.84 4.22 L 1861.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 540.4 l 3.84 4.22 L 1901.2 535.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 580.4 l 3.84 4.22 L 381.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 580.4 l 3.84 4.22 L 421.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 528.8 580.4 l 3.84 4.22 L 541.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 580.4 l 3.84 4.22 L 581.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 580.4 l 3.84 4.22 L 741.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 848.8 580.4 l 3.84 4.22 L 861.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 580.4 l 3.84 4.22 L 901.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 580.4 l 3.84 4.22 L 941.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 580.4 l 3.84 4.22 L 981.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 580.4 l 3.84 4.22 L 1021.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 580.4 l 3.84 4.22 L 1061.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 580.4 l 3.84 4.22 L 1141.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 580.4 l 3.84 4.22 L 1181.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 580.4 l 3.84 4.22 L 1221.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 580.4 l 3.84 4.22 L 1261.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 580.4 l 3.84 4.22 L 1341.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1528.8 580.4 l 3.84 4.22 L 1541.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 580.4 l 3.84 4.22 L 1581.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 580.4 l 3.84 4.22 L 1621.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 580.4 l 3.84 4.22 L 1901.2 575.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 620.4 l 3.84 4.22 L 141.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 620.4 l 3.84 4.22 L 221.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 620.4 l 3.84 4.22 L 261.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 620.4 l 3.84 4.22 L 301.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 620.4 l 3.84 4.22 L 341.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 620.4 l 3.84 4.22 L 381.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 620.4 l 3.84 4.22 L 421.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 620.4 l 3.84 4.22 L 461.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 620.4 l 3.84 4.22 L 581.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 620.4 l 3.84 4.22 L 621.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 620.4 l 3.84 4.22 L 821.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 848.8 620.4 l 3.84 4.22 L 861.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 620.4 l 3.84 4.22 L 981.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 620.4 l 3.84 4.22 L 1021.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 620.4 l 3.84 4.22 L 1061.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 620.4 l 3.84 4.22 L 1221.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 620.4 l 3.84 4.22 L 1301.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 620.4 l 3.84 4.22 L 1581.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 620.4 l 3.84 4.22 L 1621.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 620.4 l 3.84 4.22 L 1741.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 620.4 l 3.84 4.22 L 1781.2 615.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 660.4 l 3.84 4.22 L 101.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 660.4 l 3.84 4.22 L 141.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 660.4 l 3.84 4.22 L 181.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 368.8 660.4 l 3.84 4.22 L 381.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 660.4 l 3.84 4.22 L 581.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 660.4 l 3.84 4.22 L 741.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 660.4 l 3.84 4.22 L 901.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 660.4 l 3.84 4.22 L 941.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 660.4 l 3.84 4.22 L 1061.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 660.4 l 3.84 4.22 L 1221.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 660.4 l 3.84 4.22 L 1341.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 660.4 l 3.84 4.22 L 1421.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 660.4 l 3.84 4.22 L 1621.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 660.4 l 3.84 4.22 L 1741.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 660.4 l 3.84 4.22 L 1781.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 660.4 l 3.84 4.22 L 1821.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 660.4 l 3.84 4.22 L 1901.2 655.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 700.4 l 3.84 4.22 L 21.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 700.4 l 3.84 4.22 L 101.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 700.4 l 3.84 4.22 L 181.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 700.4 l 3.84 4.22 L 221.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 700.4 l 3.84 4.22 L 261.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 700.4 l 3.84 4.22 L 301.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 700.4 l 3.84 4.22 L 421.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 700.4 l 3.84 4.22 L 581.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 700.4 l 3.84 4.22 L 741.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 700.4 l 3.84 4.22 L 781.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 700.4 l 3.84 4.22 L 1061.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 700.4 l 3.84 4.22 L 1141.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 700.4 l 3.84 4.22 L 1421.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 700.4 l 3.84 4.22 L 1501.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 700.4 l 3.84 4.22 L 1661.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 700.4 l 3.84 4.22 L 1701.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 700.4 l 3.84 4.22 L 1781.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 700.4 l 3.84 4.22 L 1901.2 695.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 740.4 l 3.84 4.22 L 101.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 740.4 l 3.84 4.22 L 141.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 740.4 l 3.84 4.22 L 221.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 740.4 l 3.84 4.22 L 261.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 740.4 l 3.84 4.22 L 461.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 740.4 l 3.84 4.22 L 501.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 740.4 l 3.84 4.22 L 901.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 740.4 l 3.84 4.22 L 941.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 740.4 l 3.84 4.22 L 1061.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1128.8 740.4 l 3.84 4.22 L 1141.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 740.4 l 3.84 4.22 L 1181.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 740.4 l 3.84 4.22 L 1341.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 740.4 l 3.84 4.22 L 1421.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 740.4 l 3.84 4.22 L 1581.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1648.8 740.4 l 3.84 4.22 L 1661.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 740.4 l 3.84 4.22 L 1741.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 740.4 l 3.84 4.22 L 1781.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 740.4 l 3.84 4.22 L 1861.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 740.4 l 3.84 4.22 L 1901.2 735.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 780.4 l 3.84 4.22 L 141.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 780.4 l 3.84 4.22 L 221.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 780.4 l 3.84 4.22 L 341.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 780.4 l 3.84 4.22 L 501.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 528.8 780.4 l 3.84 4.22 L 541.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 780.4 l 3.84 4.22 L 581.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 780.4 l 3.84 4.22 L 621.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 780.4 l 3.84 4.22 L 781.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 780.4 l 3.84 4.22 L 901.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 780.4 l 3.84 4.22 L 941.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 780.4 l 3.84 4.22 L 1061.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 780.4 l 3.84 4.22 L 1181.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 780.4 l 3.84 4.22 L 1221.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 780.4 l 3.84 4.22 L 1301.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 780.4 l 3.84 4.22 L 1381.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 780.4 l 3.84 4.22 L 1421.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 780.4 l 3.84 4.22 L 1701.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 780.4 l 3.84 4.22 L 1741.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 780.4 l 3.84 4.22 L 1781.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 780.4 l 3.84 4.22 L 1861.2 775.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 820.4 l 3.84 4.22 L 21.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 820.4 l 3.84 4.22 L 101.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 820.4 l 3.84 4.22 L 141.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 820.4 l 3.84 4.22 L 181.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 820.4 l 3.84 4.22 L 221.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 820.4 l 3.84 4.22 L 341.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 820.4 l 3.84 4.22 L 421.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 448.8 820.4 l 3.84 4.22 L 461.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 820.4 l 3.84 4.22 L 621.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 820.4 l 3.84 4.22 L 661.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 820.4 l 3.84 4.22 L 701.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 820.4 l 3.84 4.22 L 821.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 848.8 820.4 l 3.84 4.22 L 861.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 820.4 l 3.84 4.22 L 901.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 820.4 l 3.84 4.22 L 1021.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 820.4 l 3.84 4.22 L 1061.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 820.4 l 3.84 4.22 L 1101.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 820.4 l 3.84 4.22 L 1261.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 820.4 l 3.84 4.22 L 1301.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 820.4 l 3.84 4.22 L 1381.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1408.8 820.4 l 3.84 4.22 L 1421.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 820.4 l 3.84 4.22 L 1501.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1528.8 820.4 l 3.84 4.22 L 1541.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 820.4 l 3.84 4.22 L 1781.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 820.4 l 3.84 4.22 L 1901.2 815.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 860.4 l 3.84 4.22 L 61.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 860.4 l 3.84 4.22 L 101.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 860.4 l 3.84 4.22 L 181.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 860.4 l 3.84 4.22 L 221.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 248.8 860.4 l 3.84 4.22 L 261.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 860.4 l 3.84 4.22 L 341.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 488.8 860.4 l 3.84 4.22 L 501.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 860.4 l 3.84 4.22 L 581.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 860.4 l 3.84 4.22 L 701.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 728.8 860.4 l 3.84 4.22 L 741.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 860.4 l 3.84 4.22 L 781.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 808.8 860.4 l 3.84 4.22 L 821.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 888.8 860.4 l 3.84 4.22 L 901.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 860.4 l 3.84 4.22 L 941.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 968.8 860.4 l 3.84 4.22 L 981.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 860.4 l 3.84 4.22 L 1221.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 860.4 l 3.84 4.22 L 1261.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 860.4 l 3.84 4.22 L 1461.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 860.4 l 3.84 4.22 L 1501.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 860.4 l 3.84 4.22 L 1741.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 860.4 l 3.84 4.22 L 1821.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 860.4 l 3.84 4.22 L 1861.2 855.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 900.4 l 3.84 4.22 L 21.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 48.8 900.4 l 3.84 4.22 L 61.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 128.8 900.4 l 3.84 4.22 L 141.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 900.4 l 3.84 4.22 L 181.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 328.8 900.4 l 3.84 4.22 L 341.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 900.4 l 3.84 4.22 L 421.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 528.8 900.4 l 3.84 4.22 L 541.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 900.4 l 3.84 4.22 L 581.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 900.4 l 3.84 4.22 L 661.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 900.4 l 3.84 4.22 L 701.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 900.4 l 3.84 4.22 L 1061.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 900.4 l 3.84 4.22 L 1101.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 900.4 l 3.84 4.22 L 1221.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 900.4 l 3.84 4.22 L 1261.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 900.4 l 3.84 4.22 L 1301.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 900.4 l 3.84 4.22 L 1461.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1528.8 900.4 l 3.84 4.22 L 1541.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1568.8 900.4 l 3.84 4.22 L 1581.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1608.8 900.4 l 3.84 4.22 L 1621.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1688.8 900.4 l 3.84 4.22 L 1701.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 900.4 l 3.84 4.22 L 1741.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 900.4 l 3.84 4.22 L 1781.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 900.4 l 3.84 4.22 L 1821.2 895.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 940.4 l 3.84 4.22 L 21.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 88.8 940.4 l 3.84 4.22 L 101.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 940.4 l 3.84 4.22 L 221.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 528.8 940.4 l 3.84 4.22 L 541.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 940.4 l 3.84 4.22 L 621.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 648.8 940.4 l 3.84 4.22 L 661.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 940.4 l 3.84 4.22 L 701.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 940.4 l 3.84 4.22 L 941.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 940.4 l 3.84 4.22 L 1021.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1048.8 940.4 l 3.84 4.22 L 1061.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1088.8 940.4 l 3.84 4.22 L 1101.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1248.8 940.4 l 3.84 4.22 L 1261.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1288.8 940.4 l 3.84 4.22 L 1301.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1368.8 940.4 l 3.84 4.22 L 1381.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 940.4 l 3.84 4.22 L 1501.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1728.8 940.4 l 3.84 4.22 L 1741.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1768.8 940.4 l 3.84 4.22 L 1781.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1808.8 940.4 l 3.84 4.22 L 1821.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 940.4 l 3.84 4.22 L 1901.2 935.35" data-astro-cid-t4uphou2=""></path><path d="M 8.8 980.4 l 3.84 4.22 L 21.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 168.8 980.4 l 3.84 4.22 L 181.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 208.8 980.4 l 3.84 4.22 L 221.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 288.8 980.4 l 3.84 4.22 L 301.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 408.8 980.4 l 3.84 4.22 L 421.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 528.8 980.4 l 3.84 4.22 L 541.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 568.8 980.4 l 3.84 4.22 L 581.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 608.8 980.4 l 3.84 4.22 L 621.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 688.8 980.4 l 3.84 4.22 L 701.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 768.8 980.4 l 3.84 4.22 L 781.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 928.8 980.4 l 3.84 4.22 L 941.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1008.8 980.4 l 3.84 4.22 L 1021.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1168.8 980.4 l 3.84 4.22 L 1181.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1208.8 980.4 l 3.84 4.22 L 1221.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1328.8 980.4 l 3.84 4.22 L 1341.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1448.8 980.4 l 3.84 4.22 L 1461.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1488.8 980.4 l 3.84 4.22 L 1501.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1848.8 980.4 l 3.84 4.22 L 1861.2 975.35" data-astro-cid-t4uphou2=""></path><path d="M 1888.8 980.4 l 3.84 4.22 L 1901.2 975.35" data-astro-cid-t4uphou2=""></path></g><g fill="none" stroke="#3F464E" stroke-width="2.7" stroke-linecap="round" data-astro-cid-t4uphou2=""><path d="M 130 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 330 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 770 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 930 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1810 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1850 15 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 90 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 170 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1850 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 55 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 10 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 330 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1370 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1770 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 95 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 10 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 50 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 770 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 135 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 490 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 930 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 175 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 330 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1770 215 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1050 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 255 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 10 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 90 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 170 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 330 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 730 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 770 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1850 295 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 90 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 490 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 930 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1370 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1850 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 335 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 50 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 90 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 170 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1170 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1850 375 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 90 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 490 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 570 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1810 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 415 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 170 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 330 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 730 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1170 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 455 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 570 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 930 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1050 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1810 495 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 490 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 770 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1770 535 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 50 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 330 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 490 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1370 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1810 575 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 10 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 730 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1370 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 615 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 50 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 210 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 655 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 50 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1170 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1250 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1810 695 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 10 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 170 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 770 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1370 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 735 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 170 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 690 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 730 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 775 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 50 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 250 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 490 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 530 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 570 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 930 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1170 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1610 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1730 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1810 815 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 410 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 450 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 610 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 850 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1090 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1570 855 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 290 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 730 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 890 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 930 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1010 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1170 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1490 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1850 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1890 895 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 570 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1170 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1210 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1330 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1450 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 935 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 130 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 370 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 650 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 730 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 810 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 970 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1130 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1290 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1370 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1410 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1530 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1650 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1690 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path><path d="M 1770 975 l 10 10 m 0 -10 l -10 10" data-astro-cid-t4uphou2=""></path></g></g></g></g><circle data-loupe="" cx="1240" cy="500" r="226" fill="none" stroke="#EEEEEE" stroke-width="12" data-astro-cid-t4uphou2=""></circle></svg></div></div><div class="post-content-shell"><article class="content"> <p>Many industries rely on relational databases that are queried with SQL. Most SQL is machine-written, but humans alone likely write billions of custom SQL queries each month<label for="sn-queries" class="sidenote-number"></label><input type="checkbox" id="sn-queries" class="margin-toggle"><span class="sidenote">Based on our internal estimates and publicly available data, such as from <a href="https://www.sec.gov/Archives/edgar/data/1640147/000164014722000023/snow-20220131.htm" rel="noopener" target="_blank">Snowflake filings</a>.</span> prompted by business questions. They are quite good at it — humans score 92.96% on <a href="https://arxiv.org/pdf/2305.03111" rel="noopener" target="_blank">BIRD</a>, a realistic benchmark for translating natural-language questions into SQL.</p> <p>However, AI performance on text-to-SQL has lagged behind. <a href="https://bird-bench.github.io/" rel="noopener" target="_blank">LLM scores on the BIRD leaderboard</a> improved from just below 70% in 2024 to 82% today. Frontier models like GPT-5.6 Sol Ultra and Claude Fable 5 can score in the mid-80s, albeit at a cost that is prohibitive for high-volume applications. This isn’t for lack of training data: SQL is widely represented in the internet content used in LLM pretraining. The challenge for AI is in navigating the ambiguous questions and highly-contextual schema that characterize real-world examples.</p> <p>A common approach for improving AI performance on tasks people understand well is building agentic scaffolding. Systems such as <a href="https://github.com/OpenHands/OpenHands" rel="noopener" target="_blank">OpenHands</a>, <a href="https://arxiv.org/pdf/2502.18864" rel="noopener" target="_blank">AI co-scientist</a>, and <a href="https://github.com/FoundationAgents/MetaGPT" rel="noopener" target="_blank">MetaGPT</a> decompose a task into stages, each with its own prompt or model call. <a href="https://link.springer.com/chapter/10.1007/978-981-92-3438-7_27" rel="noopener" target="_blank">Text-to-SQL scaffolds</a> follow the same pattern. A schema-linking stage narrows thousands of columns down to a candidate set.<label for="sn-columns" class="sidenote-number"></label><input type="checkbox" id="sn-columns" class="margin-toggle"><span class="sidenote">Real-world, enterprise data systems (including databases, data warehouses, and data lakehouses) contain up to <a href="https://www.ibm.com/think/insights/data-consumption-challenges" rel="noopener" target="_blank">millions of columns</a>. Answering business questions often involves understanding which columns to use. Academic benchmarks are simpler.</span> A generation stage samples queries. A self-correction stage repairs execution errors. A selection stage votes among the survivors. Every component is a separate call, and the orchestration is typically tuned for a given benchmark.</p> <figure id="fig:scaffolding-pipelines"><div class="flowdiagram-inline" role="img" aria-label="Flow diagram of a text-to-SQL scaffolding pipeline. A natural-language question and a database schema feed an orchestration layer, which makes separate dashed calls to stages for intent disambiguation, schema linking, query generation, refinement, and merging and selection, producing a SQL query." data-astro-cid-ckzgiyhp=""><svg viewBox="54 74 1232 702" aria-hidden="true" focusable="false" data-astro-cid-ckzgiyhp=""><g class="fd-edges" data-astro-cid-ckzgiyhp=""><path d="M 340 222 H 434" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 423 214 L 434 222 L 423 230" data-astro-cid-ckzgiyhp=""></path><path d="M 340 372 H 434" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 423 364 L 434 372 L 423 380" data-astro-cid-ckzgiyhp=""></path><path d="M 440 682 H 346" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 357 674 L 346 682 L 357 690" data-astro-cid-ckzgiyhp=""></path><path class="fd-dashed" d="M 532 125 H 714" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 703 117 L 714 125 L 703 133" data-astro-cid-ckzgiyhp=""></path><path class="fd-dashed" d="M 532 275 H 714" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 703 267 L 714 275 L 703 283" data-astro-cid-ckzgiyhp=""></path><path class="fd-dashed" d="M 532 425 H 714" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 703 417 L 714 425 L 703 433" data-astro-cid-ckzgiyhp=""></path><path class="fd-dashed" d="M 532 575 H 714" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 703 567 L 714 575 L 703 583" data-astro-cid-ckzgiyhp=""></path><path class="fd-dashed" d="M 532 725 H 714" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 703 717 L 714 725 L 703 733" data-astro-cid-ckzgiyhp=""></path><path d="M 1000 170 V 224" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 992 213 L 1000 224 L 1008 213" data-astro-cid-ckzgiyhp=""></path><path d="M 1000 320 V 374" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 992 363 L 1000 374 L 1008 363" data-astro-cid-ckzgiyhp=""></path><path d="M 1000 470 V 524" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 992 513 L 1000 524 L 1008 513" data-astro-cid-ckzgiyhp=""></path><path d="M 1000 620 V 674" data-astro-cid-ckzgiyhp=""></path><path class="fd-head" d="M 992 663 L 1000 674 L 1008 663" data-astro-cid-ckzgiyhp=""></path></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="60" y="180" width="280" height="84" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="200" y="231" data-astro-cid-ckzgiyhp="">NL question</text></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="60" y="330" width="280" height="84" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="200" y="381" data-astro-cid-ckzgiyhp="">DB schema</text></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="60" y="640" width="280" height="84" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="200" y="691" data-astro-cid-ckzgiyhp="">SQL query</text></g><g class="fd-node fd-muted" data-astro-cid-ckzgiyhp=""><rect x="440" y="90" width="92" height="680" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="486" y="439" transform="rotate(-90 486 430)" data-astro-cid-ckzgiyhp="">Orchestration</text></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="720" y="80" width="560" height="90" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="1000" y="134" data-astro-cid-ckzgiyhp="">Intent disambiguation</text></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="720" y="230" width="560" height="90" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="1000" y="284" data-astro-cid-ckzgiyhp="">Schema linking</text></g><g class="fd-node fd-accent" data-astro-cid-ckzgiyhp=""><rect x="720" y="380" width="560" height="90" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="1000" y="434" data-astro-cid-ckzgiyhp="">Query generation</text></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="720" y="530" width="560" height="90" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="1000" y="584" data-astro-cid-ckzgiyhp="">Refinement</text></g><g class="fd-node" data-astro-cid-ckzgiyhp=""><rect x="720" y="680" width="560" height="90" rx="8" data-astro-cid-ckzgiyhp=""></rect><text x="1000" y="734" data-astro-cid-ckzgiyhp="">Merging &amp; selection</text></g></svg></div><figcaption>State-of-the-art text-to-SQL scaffolding from [<a href="https://arxiv.org/pdf/2509.24403" rel="noopener" target="_blank">1</a>, <a href="https://arxiv.org/pdf/2410.01943" rel="noopener" target="_blank">2</a>, <a href="https://github.com/jd-opensource/joyagent-jdgenie/tree/data_agent" rel="noopener" target="_blank">3</a>]. The three systems differ in their component inventory but share a design premise: the model is held fixed and task performance is improved by increasing the number and structure of calls made to it.</figcaption></figure> <p>Scaffolding is an attempt to get over the limitations of model reasoning by making it adhere to a sequence of steps that mirrors how a human would approach the task. And yet, the best scaffolded models still lag 11 points behind humans on SQL. Human professionals acquire their skills with repeated experience, not by being handed a list of instructions — the same should be true of LLMs. Experience with the task should be used to train better reasoning about queries and databases into the model, instead of simply updating the prompts it receives from the scaffold.</p> <p>In this blog post we describe fine-tuning a model that achieves human-level accuracy on text-to-SQL without scaffolding, using reinforcement learning with verifiable rewards (RLVR) on <a href="https://thinkingmachines.ai/tinker/">Tinker</a>.</p> <p>Extracting the correct answer with SQL is a verifiable task that can be trained in a straightforward manner. However, the performance gap of SQL models suggested that the standard recipe can be improved. Our approach has two crucial improvements: an expert-verified training set purged of the label errors that could <a href="https://arxiv.org/abs/2603.16140" rel="noopener" target="_blank">poison RLVR</a>, and a reward-shaping technique that targets two common failure modes of RLVR in this domain.</p> <p>The trained model, <a href="https://arxiv.org/abs/2603.20004" rel="noopener" target="_blank">ReViSQL-K2.6</a>, exceeds the human mark of 92.96% when picking from 16 samples (SC-16)<label for="sn-sc16" class="sidenote-number"></label><input type="checkbox" id="sn-sc16" class="margin-toggle"><span class="sidenote">SC refers to self-consistency selection, where we group concurrently generated SQL queries by their execution results and randomly select a query from the majority group. Neither component is part of a traditional agentic scaffold: sample generation simply draws multiple outputs from the same model prompt, without separately prompted intermediate steps, while majority voting requires no additional model calls.</span> at a cost of $0.56 per task. It is more accurate than Fable 5 and GPT-5.6 Sol Ultra at 12–15% of their cost, and much more accurate than any scaffolded model on the leaderboard.</p> <figure><div class="chartscatter" role="img" aria-label="Scatter plot of Arcwise-Plat-SQL accuracy against cost per query on a logarithmic scale. ReViSQL-K2.6 reaches 91.37 percent at 3.5 cents with greedy decoding and 92.97 percent at 56 cents with 16-sample self-consistency, crossing the 92.96 percent human proxy. GPT-5.6 Sol Ultra reaches 86.75 percent at 23 cents and 89.60 percent at 3 dollars 68. Claude Fable 5 reaches 84.94 percent at 29 cents and 85.10 percent at 4 dollars 66. Five scaffolded pipelines range from 71.08 to 82.93 percent." data-astro-cid-uj4q7ace=""><svg viewBox="0 0 800 560" aria-hidden="true" focusable="false" data-astro-cid-uj4q7ace=""><g class="chartscatter-legend" data-astro-cid-uj4q7ace=""><g transform="translate(26 30)" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="-6.4" y="-6.4" width="12.8" height="12.8" transform="rotate(45 0 0)" data-astro-cid-uj4q7ace="true"></rect><text x="15" y="4" data-astro-cid-uj4q7ace="">ReViSQL-K2.6 (RLVR, single model)</text></g><g transform="translate(292.8 30)" data-astro-cid-uj4q7ace=""><circle class="chartscatter-frontier-a" cx="0" cy="0" r="7" data-astro-cid-uj4q7ace="true"></circle><text x="15" y="4" data-astro-cid-uj4q7ace="">GPT-5.6 Sol Ultra</text></g><g transform="translate(451.5 30)" data-astro-cid-uj4q7ace=""><path class="chartscatter-frontier-b" d="M 0 -8 L 7.5 6 L -7.5 6 Z" data-astro-cid-uj4q7ace="true"></path><text x="15" y="4" data-astro-cid-uj4q7ace="">Claude Fable 5</text></g><g transform="translate(590 30)" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="-6" y="-6" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect><text x="15" y="4" data-astro-cid-uj4q7ace="">Scaffolded pipelines</text></g></g><g class="chartscatter-grid" data-astro-cid-uj4q7ace=""><line x1="84" y1="401.1" x2="776" y2="401.1" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="341.3" x2="776" y2="341.3" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="281.4" x2="776" y2="281.4" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="221.6" x2="776" y2="221.6" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="161.8" x2="776" y2="161.8" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="102" x2="776" y2="102" data-astro-cid-uj4q7ace=""></line></g><g class="chartscatter-axes" data-astro-cid-uj4q7ace=""><line x1="84" y1="90" x2="84" y2="425" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="425" x2="776" y2="425" data-astro-cid-uj4q7ace=""></line></g><g class="chartscatter-y-ticks" data-astro-cid-uj4q7ace=""><text x="70" y="405.1" data-astro-cid-uj4q7ace="">70%</text><text x="70" y="345.3" data-astro-cid-uj4q7ace="">75%</text><text x="70" y="285.4" data-astro-cid-uj4q7ace="">80%</text><text x="70" y="225.6" data-astro-cid-uj4q7ace="">85%</text><text x="70" y="165.8" data-astro-cid-uj4q7ace="">90%</text><text x="70" y="106" data-astro-cid-uj4q7ace="">95%</text></g><g class="chartscatter-x-ticks" data-astro-cid-uj4q7ace=""><text x="84" y="453" data-astro-cid-uj4q7ace="">$0.001</text><text x="257" y="453" data-astro-cid-uj4q7ace="">$0.01</text><text x="430" y="453" data-astro-cid-uj4q7ace="">$0.10</text><text x="603" y="453" data-astro-cid-uj4q7ace="">$1</text><text x="776" y="453" data-astro-cid-uj4q7ace="">$10</text></g><text class="chartscatter-y-label" x="24" y="258" transform="rotate(-90 24 258)" data-astro-cid-uj4q7ace="">Arcwise-Plat-SQL accuracy (%)</text><text class="chartscatter-x-label" x="430" y="505" data-astro-cid-uj4q7ace="">Cost per final query (USD, log scale)</text><g class="chartscatter-refline" data-astro-cid-uj4q7ace=""><line x1="84" y1="126.4" x2="776" y2="126.4" data-astro-cid-uj4q7ace=""></line><text x="92" y="118.4" data-astro-cid-uj4q7ace="">Human proxy, 92.96%</text></g><path class="chartscatter-connector" d="M 351.1 145.4 L 559.4 126.3" data-astro-cid-uj4q7ace=""></path><g class="chartscatter-points" data-astro-cid-uj4q7ace=""><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="344.70000000000005" y="139" width="12.8" height="12.8" transform="rotate(45 351.1 145.4)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="351.1" y="169.4" data-astro-cid-uj4q7ace="">greedy</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="553" y="119.89999999999999" width="12.8" height="12.8" transform="rotate(45 559.4 126.3)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="569.4" y="116.3" data-astro-cid-uj4q7ace="">SC-16</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><circle class="chartscatter-frontier-a" cx="492.6" cy="200.7" r="7" data-astro-cid-uj4q7ace="true"></circle></g><text class="chartscatter-point-label chartscatter-anchor-end" x="482.6" y="188.7" data-astro-cid-uj4q7ace="">greedy</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><circle class="chartscatter-frontier-a" cx="700.9" cy="166.6" r="7" data-astro-cid-uj4q7ace="true"></circle></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="700.9" y="152.6" data-astro-cid-uj4q7ace="">SC-16</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><path class="chartscatter-frontier-b" d="M 510 214.3 L 517.5 228.3 L 502.5 228.3 Z" data-astro-cid-uj4q7ace="true"></path></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="514" y="244.3" data-astro-cid-uj4q7ace="">greedy</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><path class="chartscatter-frontier-b" d="M 718.6 212.4 L 726.1 226.4 L 711.1 226.4 Z" data-astro-cid-uj4q7ace="true"></path></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="718.6" y="244.4" data-astro-cid-uj4q7ace="">SC-16</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="380.4" y="240.4" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-end" x="374.4" y="236.4" data-astro-cid-uj4q7ace="">OpenSearch (GPT-5.2)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="389.3" y="254.7" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="407.3" y="274.7" data-astro-cid-uj4q7ace="">GenaSQL (GPT-5.2)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="524.3" y="334.1" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="542.3" y="344.1" data-astro-cid-uj4q7ace="">Contextual (XiYan-32B)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="158" y="382.2" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="164" y="412.2" data-astro-cid-uj4q7ace="">CSC-SQL (XiYan-32B)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="207.4" y="379.6" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="225.4" y="377.6" data-astro-cid-uj4q7ace="">SHARE (GPT-5.2)</text></g></g></svg></div><figcaption>Our single-model method, ReViSQL-K2.6, achieved higher accuracy than frontier models and prior open-source scaffolded pipelines on the expert-verified Arcwise-Plat-SQL benchmark. With 16-sample self-consistency (SC-16), ReViSQL-K2.6 exceeded the 92.96% human proxy for the first time.</figcaption></figure> <p>Code, data, and training recipes are at <a href="https://github.com/uiuc-kang-lab/ReViSQL" rel="noopener" target="_blank">github.com/uiuc-kang-lab/ReViSQL</a>. We further describe our methods in detail in our <a href="https://arxiv.org/abs/2603.20004" rel="noopener" target="_blank">technical report</a>.</p> <h2 id="curating-high-quality-training-data">Curating high-quality training data</h2> <p>RLVR is effective at improving domain-specific reasoning, but it is sensitive to data with incorrect labels. In RLVR, the scalar reward is the entire learning signal for the training step. Mislabeled instances reverse the signal, degrading learning significantly. Our research found that <a href="https://ddkang.substack.com/p/noisy-data-breaks-rlvr" rel="noopener" target="_blank">algorithmic tweaks cannot compensate for this loss</a> — cleaning up the data is crucial for RLVR to work well.</p> <p>We found existing text-to-SQL data to be extremely noisy.<label for="sn-noise" class="sidenote-number"></label><input type="checkbox" id="sn-noise" class="margin-toggle"><span class="sidenote">The noisiness of public SQL datasets was noted by many others such as <a href="https://arxiv.org/abs/2310.18538" rel="noopener" target="_blank">Pourreza and Rafiei (2023)</a> and <a href="https://arxiv.org/abs/2402.12243" rel="noopener" target="_blank">Wretblad et al. (2024)</a>. <a href="https://arxiv.org/abs/2507.02825" rel="noopener" target="_blank">Our analysis</a> has shown that many widely known benchmarks contain large amounts of noise.</span> We sampled 2.5k instances from BIRD Train, a training dataset for text-to-SQL. Our audit found errors in every component of the dataset: the questions, the external knowledge supplied, and in more than half of the “golden SQL queries” that the model’s answer is compared against.</p> <table><thead><tr><th>Error type</th><th>Share of audited instances</th></tr></thead><tbody><tr><td>Gold SQL query incorrect</td><td>52.1%</td></tr><tr><td>Natural language question flawed</td><td>26.2%</td></tr><tr><td>External knowledge entry wrong</td><td>18.2%</td></tr><tr><td>Unanswerable given the schema (discarded)</td><td>1.5%</td></tr><tr><td>At least one of the above</td><td>61.1%</td></tr></tbody></table> <div class="table-caption" data-astro-cid-eafoyku3=""><span class="table-caption-label" data-astro-cid-eafoyku3="">Table 1:</span> Annotation error rates across 2.5k instances sampled from BIRD Train. Categories overlap, so the total is not a sum.</div> <p>We cleaned up the training set in a multi-stage process. First, an LLM (OpenAI’s o3) and a human expert reviewed each instance and flagged errors. The expert review found that the LLM auditor was precise in catching annotation mistakes (90.6% precision) but only caught 24.5% of the errors flagged by humans. The errors and proposed fixes from this first stage were sent to a different expert for verification. Where the verifier disagreed with the initial auditor, the sample was sent back for additional loops of conflict resolution.</p> <p>We released the cleaned-up training set to the community as <a href="https://github.com/uiuc-kang-lab/ReViSQL/blob/main/data/bird-plat-2.5k-v1.json" rel="noopener" target="_blank">BIRD-Platinum</a>.</p> <figure id="fig:bird-platinum-pipeline"><div class="flowdiagram-inline" role="img" aria-label="Flow diagram of the BIRD-Platinum correction pipeline. The original BIRD Train dataset flows into a correction stage, then a verification stage, with a dashed conflict-resolution loop sending verifier disagreements back for another round, and emerges as the BIRD-Platinum dataset." data-astro-cid-lbfoo57t=""><svg viewBox="44 64 1522 572" aria-hidden="true" focusable="false" data-astro-cid-lbfoo57t=""><rect class="fd-frame" x="440" y="70" width="730" height="560" rx="18" data-astro-cid-lbfoo57t=""></rect><g class="fd-edges" data-astro-cid-lbfoo57t=""><path d="M 340 295 H 514" data-astro-cid-lbfoo57t=""></path><path class="fd-head" d="M 503 287 L 514 295 L 503 303" data-astro-cid-lbfoo57t=""></path><path d="M 770 215 H 834" data-astro-cid-lbfoo57t=""></path><path class="fd-head" d="M 823 207 L 834 215 L 823 223" data-astro-cid-lbfoo57t=""></path><path d="M 1090 295 H 1254" data-astro-cid-lbfoo57t=""></path><path class="fd-head" d="M 1243 287 L 1254 295 L 1243 303" data-astro-cid-lbfoo57t=""></path><path class="fd-dashed" d="M 930 260 V 454" data-astro-cid-lbfoo57t=""></path><path class="fd-head" d="M 922 443 L 930 454 L 938 443" data-astro-cid-lbfoo57t=""></path><path class="fd-dashed" d="M 670 460 V 266" data-astro-cid-lbfoo57t=""></path><path class="fd-head" d="M 662 277 L 670 266 L 678 277" data-astro-cid-lbfoo57t=""></path></g><g class="fd-node fd-muted" data-astro-cid-lbfoo57t=""><rect x="50" y="240" width="290" height="110" rx="8" data-astro-cid-lbfoo57t=""></rect><g class="fd-icon" transform="translate(83 275)" data-astro-cid-lbfoo57t=""><svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" class="astro-lbfoo57t" data-astro-cid-lbfoo57t=""><ellipse cx="12" cy="5" rx="9" ry="3"></ellipse><path d="M3 5V19A9 3 0 0 0 21 19V5"></path><path d="M3 12A9 3 0 0 0 21 12"></path></svg></g><text x="215" y="287" data-astro-cid-lbfoo57t="">Original</text><text x="215" y="323" data-astro-cid-lbfoo57t="">BIRD Train</text></g><g class="fd-node" data-astro-cid-lbfoo57t=""><rect x="500" y="170" width="270" height="90" rx="8" data-astro-cid-lbfoo57t=""></rect><g class="fd-icon" transform="translate(528 195)" data-astro-cid-lbfoo57t=""><svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" class="astro-lbfoo57t astro-lbfoo57t" data-astro-cid-lbfoo57t=""><path d="M16 4h2a2 2 0 0 1 2 2v2"></path><path d="M21.34 15.664a1 1 0 1 0-3.004-3.004l-5.01 5.012a2 2 0 0 0-.506.854l-.837 2.87a.5.5 0 0 0 .62.62l2.87-.837a2 2 0 0 0 .854-.506z"></path><path d="M8 22H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2"></path><rect x="8" y="2" width="8" height="4" rx="1"></rect></svg></g><text x="655" y="226" data-astro-cid-lbfoo57t="">Correction</text></g><g class="fd-node" data-astro-cid-lbfoo57t=""><rect x="840" y="170" width="290" height="90" rx="8" data-astro-cid-lbfoo57t=""></rect><g class="fd-icon" transform="translate(866 195)" data-astro-cid-lbfoo57t=""><svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" class="astro-lbfoo57t astro-lbfoo57t astro-lbfoo57t" data-astro-cid-lbfoo57t=""><rect width="8" height="4" x="8" y="2" rx="1" ry="1"></rect><path d="M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2"></path><path d="m9 14 2 2 4-4"></path></svg></g><text x="1000" y="226" data-astro-cid-lbfoo57t="">Verification</text></g><g class="fd-node fd-muted" data-astro-cid-lbfoo57t=""><rect x="600" y="460" width="400" height="90" rx="8" data-astro-cid-lbfoo57t=""></rect><g class="fd-icon" transform="translate(630 485)" data-astro-cid-lbfoo57t=""><svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" class="astro-lbfoo57t astro-lbfoo57t astro-lbfoo57t astro-lbfoo57t" data-astro-cid-lbfoo57t=""><path d="M12 3v18"></path><path d="m19 8 3 8a5 5 0 0 1-6 0zV7"></path><path d="M3 7h1a17 17 0 0 0 8-2 17 17 0 0 0 8 2h1"></path><path d="m5 8 3 8a5 5 0 0 1-6 0zV7"></path><path d="M7 21h10"></path></svg></g><text x="822" y="516" data-astro-cid-lbfoo57t="">Conflict resolution</text></g><g class="fd-node fd-accent" data-astro-cid-lbfoo57t=""><rect x="1260" y="240" width="300" height="110" rx="8" data-astro-cid-lbfoo57t=""></rect><g class="fd-icon" transform="translate(1282 275)" data-astro-cid-lbfoo57t=""><svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" class="astro-lbfoo57t astro-lbfoo57t astro-lbfoo57t astro-lbfoo57t astro-lbfoo57t" data-astro-cid-lbfoo57t=""><path d="m16 19 2 2 4-4"></path><path d="M21 13.127V5"></path><path d="M3 12A9 3 0 0 0 21 12"></path><path d="M3 5V19A9 3 0 0 0 13.318 21.968"></path><ellipse cx="12" cy="5" rx="9" ry="3"></ellipse></svg></g><text x="1438" y="307" data-astro-cid-lbfoo57t="">BIRD-Platinum</text></g></svg></div><figcaption>The BIRD-Platinum data correction pipeline with human experts.</figcaption></figure> <p>We suspected that the evaluation dataset, BIRD Mini-Dev, likewise contained annotation errors. A first cleanup pass of BIRD Mini-Dev was done by <a href="https://github.com/AlibabaResearch/DAMO-ConvAI/issues/39#issuecomment-2303283506" rel="noopener" target="_blank">Arcwise</a> and corrected errors in 32.3% of instances. We did a second pass that confirmed the vast majority of Arcwise’s flags and found many more, bringing the total detected error rate in BIRD Mini-Dev to 52.8%. The evaluation set with the gold query errors cleaned up was released as <a href="https://www.vldb.org/pvldb/vol19/p931-jin.pdf" rel="noopener" target="_blank">Arcwise-Plat-SQL</a>.</p> <h2 id="bird-platinum-lifts-rlvr-above-prior-best-models">BIRD-Platinum lifts RLVR above prior best models</h2> <p>We fine-tuned Kimi-K2.6 with RLVR on BIRD-Platinum to produce ReViSQL-K2.6. Training on verified data alone lifted ReViSQL-K2.6 well above both frontier generalist LLMs and the leading open-weight fine-tuned text-to-SQL models on Arcwise-Plat-SQL, with an accuracy score of 88.55%. This shows that the annotation errors in standard training data were the binding constraint on RLVR for text-to-SQL.</p> <figure><div class="chartbar" role="img" aria-label="Bar chart of pass at 1 accuracy on Arcwise-Plat-SQL for seven single models. ReViSQL-K2.6 scores 88.55 percent, GPT-5.6 Sol Ultra 86.75, Claude Fable 5 84.94, Infly-RL-SQL-32B 69.88, OmniSQL-32B 69.48, XiYanSQL-32B 68.67, and Arctic-R1-7B 67.47." data-astro-cid-oeaeatqp=""><svg viewBox="0 0 800 460" aria-hidden="true" focusable="false" data-astro-cid-oeaeatqp=""><g class="chartbar-grid" data-astro-cid-oeaeatqp=""><line x1="84" y1="360" x2="788" y2="360" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="298" x2="788" y2="298" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="236" x2="788" y2="236" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="174" x2="788" y2="174" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="112" x2="788" y2="112" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="50" x2="788" y2="50" data-astro-cid-oeaeatqp=""></line></g><g class="chartbar-ticks" data-astro-cid-oeaeatqp=""><text x="70" y="364" data-astro-cid-oeaeatqp="">0%</text><text x="70" y="302" data-astro-cid-oeaeatqp="">20%</text><text x="70" y="240" data-astro-cid-oeaeatqp="">40%</text><text x="70" y="178" data-astro-cid-oeaeatqp="">60%</text><text x="70" y="116" data-astro-cid-oeaeatqp="">80%</text><text x="70" y="54" data-astro-cid-oeaeatqp="">100%</text></g><g class="chartbar-axes" data-astro-cid-oeaeatqp=""><line x1="84" y1="50" x2="84" y2="360" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="360" x2="788" y2="360" data-astro-cid-oeaeatqp=""></line></g><text class="chartbar-y-label" x="24" y="205" transform="rotate(-90 24 205)" data-astro-cid-oeaeatqp="">Pass@1 accuracy (%)</text><g class="chartbar-bars" data-astro-cid-oeaeatqp=""><g data-category="ReViSQL-K2.6" data-astro-cid-oeaeatqp=""><rect class="chartbar-ours" x="103.1" y="85.5" width="62.4" height="274.5" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="134.3" y="77.5" data-astro-cid-oeaeatqp="">88.55</text><text class="chartbar-name chartbar-name-angled" x="134.3" y="376" transform="rotate(-24 134.3 376)" data-astro-cid-oeaeatqp="">ReViSQL-K2.6</text></g><g data-category="GPT-5.6 Sol Ultra" data-astro-cid-oeaeatqp=""><rect class="chartbar-base" x="203.7" y="91.1" width="62.4" height="268.9" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="234.9" y="83.1" data-astro-cid-oeaeatqp="">86.75</text><text class="chartbar-name chartbar-name-angled" x="234.9" y="376" transform="rotate(-24 234.9 376)" data-astro-cid-oeaeatqp="">GPT-5.6 Sol Ultra</text></g><g data-category="Claude Fable 5" data-astro-cid-oeaeatqp=""><rect class="chartbar-base" x="304.3" y="96.7" width="62.4" height="263.3" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="335.4" y="88.7" data-astro-cid-oeaeatqp="">84.94</text><text class="chartbar-name chartbar-name-angled" x="335.4" y="376" transform="rotate(-24 335.4 376)" data-astro-cid-oeaeatqp="">Claude Fable 5</text></g><g data-category="Infly-RL-SQL-32B" data-astro-cid-oeaeatqp=""><rect class="chartbar-base" x="404.8" y="143.4" width="62.4" height="216.6" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="436" y="135.4" data-astro-cid-oeaeatqp="">69.88</text><text class="chartbar-name chartbar-name-angled" x="436" y="376" transform="rotate(-24 436 376)" data-astro-cid-oeaeatqp="">Infly-RL-SQL-32B</text></g><g data-category="OmniSQL-32B" data-astro-cid-oeaeatqp=""><rect class="chartbar-base" x="505.4" y="144.6" width="62.4" height="215.4" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="536.6" y="136.6" data-astro-cid-oeaeatqp="">69.48</text><text class="chartbar-name chartbar-name-angled" x="536.6" y="376" transform="rotate(-24 536.6 376)" data-astro-cid-oeaeatqp="">OmniSQL-32B</text></g><g data-category="XiYanSQL-32B" data-astro-cid-oeaeatqp=""><rect class="chartbar-base" x="606" y="147.1" width="62.4" height="212.9" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="637.1" y="139.1" data-astro-cid-oeaeatqp="">68.67</text><text class="chartbar-name chartbar-name-angled" x="637.1" y="376" transform="rotate(-24 637.1 376)" data-astro-cid-oeaeatqp="">XiYanSQL-32B</text></g><g data-category="Arctic-R1-7B" data-astro-cid-oeaeatqp=""><rect class="chartbar-base" x="706.5" y="150.8" width="62.4" height="209.2" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-value" x="737.7" y="142.8" data-astro-cid-oeaeatqp="">67.47</text><text class="chartbar-name chartbar-name-angled" x="737.7" y="376" transform="rotate(-24 737.7 376)" data-astro-cid-oeaeatqp="">Arctic-R1-7B</text></g></g></svg></div><figcaption>ReViSQL-K2.6, fine-tuned on BIRD-Platinum, achieves the highest accuracy on an expert-verified variant of BIRD, outperforming GPT-5.6 Sol Ultra, Claude Fable 5, and the strongest open-weight fine-tuned text-to-SQL models (<a href="https://huggingface.co/infly/inf-rl-qwen-coder-32b-2746" rel="noopener" target="_blank">Infly-RL-SQL-32B</a>, <a href="https://huggingface.co/seeklhy/OmniSQL-32B" rel="noopener" target="_blank">OmniSQL-32B</a>, <a href="https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-32B-2504" rel="noopener" target="_blank">XiYanSQL-32B</a>, <a href="https://huggingface.co/Snowflake/Arctic-Text2SQL-R1-7B" rel="noopener" target="_blank">Arctic-R1-7B</a>).</figcaption></figure> <p>To demonstrate that this approach generalizes to other models and eval sets we didn’t touch, we fine-tuned Qwen3-235B-A22B with RLVR on BIRD-Platinum and the original BIRD Train. We tested the model on two new text-to-SQL benchmarks, widely considered more difficult than BIRD:</p> <ul> <li><a href="https://arxiv.org/abs/2503.02240" rel="noopener" target="_blank"><strong>Spider2-SQLite</strong></a>: a variant of the <a href="https://arxiv.org/abs/2411.07763" rel="noopener" target="_blank">Spider2</a> benchmark that contains complex queries with 5.2× as many tokens on average as Arcwise-Plat-SQL.</li> <li><a href="https://arxiv.org/abs/2411.07763" rel="noopener" target="_blank"><strong>Spider2-Snow</strong></a>: a variant of Spider2 that uses the <a href="https://docs.snowflake.com/en/sql-reference-commands" rel="noopener" target="_blank">Snowflake SQL dialect</a>.</li> </ul> <figure><div class="chartbar" role="img" aria-label="Grouped bar chart comparing Qwen3-235B-A22B fine-tuned on BIRD-Platinum versus the original BIRD Train across three benchmarks. BIRD-Platinum scores 87.35 percent on Arcwise-Plat-SQL versus 71.10, 37.04 on Spider2-SQLite versus 25.13, and 32.00 on Spider2-Snow versus 18.10." data-astro-cid-oeaeatqp=""><svg viewBox="0 0 800 460" aria-hidden="true" focusable="false" data-astro-cid-oeaeatqp=""><g class="chartbar-legend" data-astro-cid-oeaeatqp=""><g transform="translate(240 24)" data-astro-cid-oeaeatqp=""><rect class="chartbar-ours" x="-7" y="-7" width="14" height="14" data-astro-cid-oeaeatqp=""></rect><text x="15" y="4" data-astro-cid-oeaeatqp="">BIRD-Platinum</text></g><g transform="translate(440 24)" data-astro-cid-oeaeatqp=""><rect class="chartbar-light" x="-7" y="-7" width="14" height="14" data-astro-cid-oeaeatqp=""></rect><text x="15" y="4" data-astro-cid-oeaeatqp="">BIRD Train</text></g></g><g class="chartbar-grid" data-astro-cid-oeaeatqp=""><line x1="84" y1="360" x2="788" y2="360" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="298" x2="788" y2="298" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="236" x2="788" y2="236" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="174" x2="788" y2="174" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="112" x2="788" y2="112" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="50" x2="788" y2="50" data-astro-cid-oeaeatqp=""></line></g><g class="chartbar-ticks" data-astro-cid-oeaeatqp=""><text x="70" y="364" data-astro-cid-oeaeatqp="">0%</text><text x="70" y="302" data-astro-cid-oeaeatqp="">20%</text><text x="70" y="240" data-astro-cid-oeaeatqp="">40%</text><text x="70" y="178" data-astro-cid-oeaeatqp="">60%</text><text x="70" y="116" data-astro-cid-oeaeatqp="">80%</text><text x="70" y="54" data-astro-cid-oeaeatqp="">100%</text></g><g class="chartbar-axes" data-astro-cid-oeaeatqp=""><line x1="84" y1="50" x2="84" y2="360" data-astro-cid-oeaeatqp=""></line><line x1="84" y1="360" x2="788" y2="360" data-astro-cid-oeaeatqp=""></line></g><text class="chartbar-y-label" x="24" y="205" transform="rotate(-90 24 205)" data-astro-cid-oeaeatqp="">Pass@1 accuracy (%)</text><g class="chartbar-bars" data-astro-cid-oeaeatqp=""><g data-category="Arcwise-Plat-SQL" data-astro-cid-oeaeatqp=""><rect class="chartbar-ours" x="124.9" y="89.2" width="70.4" height="270.8" data-astro-cid-oeaeatqp=""></rect><rect class="chartbar-light" x="207.3" y="139.6" width="70.4" height="220.4" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-name" x="201.3" y="384" data-astro-cid-oeaeatqp="">Arcwise-Plat-SQL</text></g><g data-category="Spider2-SQLite" data-astro-cid-oeaeatqp=""><rect class="chartbar-ours" x="359.6" y="245.2" width="70.4" height="114.8" data-astro-cid-oeaeatqp=""></rect><rect class="chartbar-light" x="442" y="282.1" width="70.4" height="77.9" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-name" x="436" y="384" data-astro-cid-oeaeatqp="">Spider2-SQLite</text></g><g data-category="Spider2-Snow" data-astro-cid-oeaeatqp=""><rect class="chartbar-ours" x="594.3" y="260.8" width="70.4" height="99.2" data-astro-cid-oeaeatqp=""></rect><rect class="chartbar-light" x="676.7" y="303.9" width="70.4" height="56.1" data-astro-cid-oeaeatqp=""></rect><text class="chartbar-name" x="670.7" y="384" data-astro-cid-oeaeatqp="">Spider2-Snow</text></g></g></svg></div><figcaption>Qwen3-235B-A22B fine-tuned on BIRD-Platinum outperforms the same model trained on the original BIRD Train across Arcwise-Plat-SQL, Spider2-SQLite, and Spider2-Snow by 16%, 12%, and 14%, respectively.</figcaption></figure> <p>Training on the more carefully curated BIRD-Platinum improves the model’s accuracy by 16% on Arcwise-Plat-SQL, 12% on Spider2-SQLite, and 14% on Spider2-Snow compared to BIRD Train. This indicates that our verified data produces a more transferable learning signal across benchmarks and SQL dialects.</p> <h2 id="accurate-reward-signal-for-text-to-sql-rlvr">Accurate reward signal for text-to-SQL RLVR</h2> <p>Training on clean data brought the fine-tuned model closer to human parity, but a gap of over 4% remained. We looked at the cases where the model failed to identify patterns, which led us to examine the reward function used in training.</p> <p>Standard text-to-SQL RLVR assigns a reward of 1 when the generated query returns the same result as the gold query on the benchmark database. This mirrors the scoring used in evaluation, but doesn’t fully capture the general behavior we want the model to learn. We focused on two ways result-based reward can diverge from the intended behavior and amended the reward function to address them.</p> <h3 id="divergence-1-execution-match-is-not-semantic-equivalence">Divergence 1: execution match is not semantic equivalence</h3> <p>The standard result-based reward checks the generated query’s output on a single database instance, but this doesn’t guarantee that the result would hold for a different one. A wrong join key or a dropped predicate can be missed when the particular database instance does not expose the error. Only queries that are semantically equivalent are guaranteed to produce the same result on any database instance.</p> <p>We test for the semantic equivalence of SQL queries with <a href="https://arxiv.org/abs/2403.03193" rel="noopener" target="_blank">VeriEQL</a>, a solver that uses bounded verification and incurs negligible CPU costs relative to the total training costs (less than 0.1%). In a pilot training run, we found that 32.8% of positive result-based rewards were given to queries that weren’t fully equivalent to the correct one. That means that nearly one time in three, the reward reinforced the wrong query.</p> <figure id="fig:failure-mode-1"><div class="revisql-failure" role="group" aria-label="A question with two SQL queries: the gold query and an incorrect query that still received a positive reward." data-astro-cid-lvznotu6=""><div class="revisql-failure-question" data-astro-cid-lvznotu6=""><b data-astro-cid-lvznotu6="">Question:</b> How much money on average does Lucas Wyldbore spend on book orders?</div><div class="revisql-failure-panels" data-astro-cid-lvznotu6=""><article class="revisql-failure-panel" data-astro-cid-lvznotu6=""><div class="revisql-failure-panel-label" data-astro-cid-lvznotu6="">Gold SQL query</div><div class="revisql-failure-sql" data-astro-cid-lvznotu6="">SELECT AVG<span class="dim" data-astro-cid-lvznotu6="">(</span><span class="mark" data-astro-cid-lvznotu6="">order_total</span><span class="dim" data-astro-cid-lvznotu6="">)</span> FROM <span class="dim" data-astro-cid-lvznotu6="">(</span> SELECT o.order_id<span class="dim" data-astro-cid-lvznotu6="">,</span> SUM<span class="dim" data-astro-cid-lvznotu6="">(</span>i.price<span class="dim" data-astro-cid-lvznotu6="">)</span> AS order_total FROM orders o JOIN items i ON o.order_id = i.order_id WHERE o.customer = 'Lucas...' GROUP BY o.order_id <span class="dim" data-astro-cid-lvznotu6="">);</span></div></article><article class="revisql-failure-panel revisql-failure-panel-bad" data-astro-cid-lvznotu6=""><div class="revisql-failure-panel-label" data-astro-cid-lvznotu6="">Incorrect query, positive reward</div><div class="revisql-failure-sql" data-astro-cid-lvznotu6="">SELECT AVG<span class="dim" data-astro-cid-lvznotu6="">(</span><span class="mark" data-astro-cid-lvznotu6="">i.price</span><span class="dim" data-astro-cid-lvznotu6="">)</span> FROM orders o JOIN items i ON o.order_id = i.order_id WHERE o.customer = 'Lucas...'</div><div class="revisql-failure-note" data-astro-cid-lvznotu6="">Averages line-item prices, not order totals. The two coincide only because each order has a single line item.</div></article></div></div><figcaption>A false-positive reward. The generated query averages line-item prices instead of order totals, yet matches the gold result because each order has a single line item. Result-based grading rewards it as correct.</figcaption></figure> <p>We updated the reward signal by downweighting it in cases where the query was accepted by execution matching but wasn’t equivalent according to VeriEQL. With an additional source of verification, the reward pushes training towards the correct query that generalizes to different database instances.</p> <h3 id="divergence-2-outcome-rewards-are-blind-to-provided-knowledge">Divergence 2: outcome rewards are blind to provided knowledge</h3> <p>BIRD-style problems provide external knowledge alongside the question in a prompt. A result-based reward conditions only on the final result, which means it cannot distinguish a model that read the provided information from one that guessed correctly based on its pretraining priors. For example, because “sodium = 0” and “sodium &lt; 5” produce the same result set, a result-based reward cannot distinguish a model that correctly uses external knowledge to choose “sodium = 0” from one that memorizes or hallucinates “sodium &lt; 5.” In the absence of a gradient pushing models to incorporate external knowledge, they tend to default to the prior. In a pilot analysis on a validation set, 24.2% of failures were traced to the model ignoring the necessary information that was supplied.</p> <figure id="fig:failure-mode-2"><div class="revisql-failure" role="group" aria-label="A question, an external-knowledge entry, and two SQL queries: the gold query, which applies the provided definition, and a generated query that ignores it." data-astro-cid-lvznotu6=""><div class="revisql-failure-question" data-astro-cid-lvznotu6=""><b data-astro-cid-lvznotu6="">Question:</b> Among recipes from The California Tree Fruit Agreement, calculate the percentage of sodium-free recipes.</div><div class="revisql-failure-question" data-astro-cid-lvznotu6=""><b data-astro-cid-lvznotu6="">External knowledge:</b> sodium-free refers to sodium = 0.</div><div class="revisql-failure-panels" data-astro-cid-lvznotu6=""><article class="revisql-failure-panel" data-astro-cid-lvznotu6=""><div class="revisql-failure-panel-label" data-astro-cid-lvznotu6="">Gold SQL query</div><div class="revisql-failure-sql" data-astro-cid-lvznotu6="">SELECT CAST<span class="dim" data-astro-cid-lvznotu6="">(</span>SUM<span class="dim" data-astro-cid-lvznotu6="">(</span> CASE WHEN <span class="mark" data-astro-cid-lvznotu6="">sodium = 0</span> THEN 1 ELSE 0 END <span class="dim" data-astro-cid-lvznotu6="">) </span>AS REAL<span class="dim" data-astro-cid-lvznotu6="">)</span> * 100 / COUNT<span class="dim" data-astro-cid-lvznotu6="">(</span>*<span class="dim" data-astro-cid-lvznotu6="">)</span> FROM recipes r WHERE r.source = 'California...'</div></article><article class="revisql-failure-panel revisql-failure-panel-bad" data-astro-cid-lvznotu6=""><div class="revisql-failure-panel-label" data-astro-cid-lvznotu6="">Ignores external knowledge</div><div class="revisql-failure-sql" data-astro-cid-lvznotu6="">SELECT CAST<span class="dim" data-astro-cid-lvznotu6="">(</span>SUM<span class="dim" data-astro-cid-lvznotu6="">(</span> CASE WHEN <span class="mark" data-astro-cid-lvznotu6="">sodium &lt; 5</span> THEN 1 ELSE 0 END <span class="dim" data-astro-cid-lvznotu6="">) </span>AS REAL<span class="dim" data-astro-cid-lvznotu6="">)</span> * 100 / COUNT<span class="dim" data-astro-cid-lvznotu6="">(</span>*<span class="dim" data-astro-cid-lvznotu6="">)</span> FROM recipes r WHERE r.source = 'California...'</div></article></div></div><figcaption>A failure from ignoring external knowledge. The external knowledge specifies that “sodium-free” means sodium = 0, but the generated query applies the threshold sodium &lt; 5 instead. Because outcome-based rewards condition only on the final result, they provide no signal that the model disregarded the provided evidence and wrote a semantically incorrect SQL query.</figcaption></figure> <p>We address this with rule-based process rewards. The model must emit a requirement block that translates each external-knowledge entry into an explicit query constraint, and a verification block that audits the generated query against those constraints, with penalties for non-compliance. Our process reward encourages the model to ground its reasoning and generation in the provided knowledge rather than blindly relying on its pre-trained prior. The rewards are rule-based rather than model-graded, making the scoring cheap and free from contamination by a judge model’s priors.</p> <h2 id="training-recipe">Training recipe</h2> <p>We provide our training recipe for ReViSQL-K2.6. This recipe can also be reproduced using our <a href="https://github.com/uiuc-kang-lab/ReViSQL" rel="noopener" target="_blank">code built on the Tinker APIs</a>.</p> <table><thead><tr><th>Component</th><th>Setting</th></tr></thead><tbody><tr><td>Base model</td><td>moonshotai/Kimi-K2.6</td></tr><tr><td>Training data</td><td><a href="https://github.com/uiuc-kang-lab/ReViSQL/blob/main/data/bird-plat-2.5k-v1.json" rel="noopener" target="_blank">BIRD-Platinum</a></td></tr><tr><td>Train/validation split</td><td>85:15</td></tr><tr><td>RL objective</td><td>CISPO</td></tr><tr><td>Batch size</td><td>64</td></tr><tr><td>Group size</td><td>16</td></tr><tr><td>Learning rate</td><td>5×10<sup>−5</sup></td></tr><tr><td>LoRA rank</td><td>32</td></tr><tr><td>Maximum number of input tokens</td><td>32,768</td></tr><tr><td>Maximum interaction turns between the model and the environment (a database instance)</td><td>5</td></tr><tr><td>Maximum number of output tokens per turn</td><td>3,072</td></tr><tr><td>VeriEQL reward shaping</td><td>An execution match refuted by VeriEQL receives a penalty of 0.2</td></tr><tr><td>Process reward shaping</td><td>Non-compliance with the required external knowledge analysis receives a penalty of 0.1</td></tr><tr><td>Checkpoint selection</td><td>Highest validation accuracy</td></tr></tbody></table> <div class="table-caption" data-astro-cid-eafoyku3=""><span class="table-caption-label" data-astro-cid-eafoyku3="">Table 2:</span> Training configuration.</div> <h2 id="results">Results</h2> <p>We present ReViSQL-K2.6, the model fine-tuned on Tinker with verified data and both reward modifications. Under greedy decoding (single sample, temperature = 0) our model achieves an accuracy of 91.37% on Arcwise-Plat-SQL at a cost of $0.035 per task. This is an 8.4 point improvement over <a href="https://arxiv.org/abs/2502.14913" rel="noopener" target="_blank">OpenSearch</a>, the strongest prior open-source pipeline, at a 37% lower cost. The cost advantage is a direct consequence of removing auxiliary scaffolding around the model.</p> <p>If ReViSQL-K2.6 votes among 16 candidates generated with temperature = 1, accuracy rises to 92.97% at a cost of $0.56 per task. This is the first time a text-to-SQL AI system has exceeded the human benchmark to our knowledge.</p> <figure><div class="chartscatter" role="img" aria-label="Scatter plot of accuracy against cost per query on a logarithmic scale. ReViSQL-K2.6 scores 91.37 percent with 1 sample at 3.5 cents, 92.57 with 4 samples at 14 cents, 92.77 with 8 samples at 28 cents, 92.97 with 16 samples at 56 cents, and 93.17 with 32 samples at 1 dollar 12. Two ablations score lower: verified data without reward shaping reaches 88.55 percent, and the original BIRD Train with reward shaping reaches 74.30 percent. Five scaffolded pipelines score between 71.08 and 82.93 percent. A dashed line marks the 92.96 percent human proxy." data-astro-cid-uj4q7ace=""><svg viewBox="0 0 800 560" aria-hidden="true" focusable="false" data-astro-cid-uj4q7ace=""><g class="chartscatter-legend" data-astro-cid-uj4q7ace=""><g transform="translate(26 30)" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="-6.4" y="-6.4" width="12.8" height="12.8" transform="rotate(45 0 0)" data-astro-cid-uj4q7ace="true"></rect><text x="15" y="4" data-astro-cid-uj4q7ace="">ReViSQL-K2.6, by samples drawn (SC)</text></g><g transform="translate(306.3 30)" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours-light" x="-6.4" y="-6.4" width="12.8" height="12.8" transform="rotate(45 0 0)" data-astro-cid-uj4q7ace="true"></rect><text x="15" y="4" data-astro-cid-uj4q7ace="">RLVR ablations (greedy)</text></g><g transform="translate(505.5 30)" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="-6" y="-6" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect><text x="15" y="4" data-astro-cid-uj4q7ace="">Scaffolded pipelines</text></g></g><g class="chartscatter-grid" data-astro-cid-uj4q7ace=""><line x1="84" y1="401.1" x2="776" y2="401.1" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="341.3" x2="776" y2="341.3" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="281.4" x2="776" y2="281.4" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="221.6" x2="776" y2="221.6" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="161.8" x2="776" y2="161.8" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="102" x2="776" y2="102" data-astro-cid-uj4q7ace=""></line></g><g class="chartscatter-axes" data-astro-cid-uj4q7ace=""><line x1="84" y1="90" x2="84" y2="425" data-astro-cid-uj4q7ace=""></line><line x1="84" y1="425" x2="776" y2="425" data-astro-cid-uj4q7ace=""></line></g><g class="chartscatter-y-ticks" data-astro-cid-uj4q7ace=""><text x="70" y="405.1" data-astro-cid-uj4q7ace="">70%</text><text x="70" y="345.3" data-astro-cid-uj4q7ace="">75%</text><text x="70" y="285.4" data-astro-cid-uj4q7ace="">80%</text><text x="70" y="225.6" data-astro-cid-uj4q7ace="">85%</text><text x="70" y="165.8" data-astro-cid-uj4q7ace="">90%</text><text x="70" y="106" data-astro-cid-uj4q7ace="">95%</text></g><g class="chartscatter-x-ticks" data-astro-cid-uj4q7ace=""><text x="84" y="453" data-astro-cid-uj4q7ace="">$0.001</text><text x="257" y="453" data-astro-cid-uj4q7ace="">$0.01</text><text x="430" y="453" data-astro-cid-uj4q7ace="">$0.10</text><text x="603" y="453" data-astro-cid-uj4q7ace="">$1</text><text x="776" y="453" data-astro-cid-uj4q7ace="">$10</text></g><text class="chartscatter-y-label" x="24" y="258" transform="rotate(-90 24 258)" data-astro-cid-uj4q7ace="">Arcwise-Plat-SQL accuracy (%)</text><text class="chartscatter-x-label" x="430" y="505" data-astro-cid-uj4q7ace="">Cost per final query (USD, log scale)</text><g class="chartscatter-refline" data-astro-cid-uj4q7ace=""><line x1="84" y1="126.4" x2="776" y2="126.4" data-astro-cid-uj4q7ace=""></line><text x="92" y="118.4" data-astro-cid-uj4q7ace="">Human proxy, 92.96%</text></g><path class="chartscatter-connector" d="M 351.1 145.4 L 455.3 131 L 507.4 128.6 L 559.4 126.3 L 611.5 123.9" data-astro-cid-uj4q7ace=""></path><g class="chartscatter-points" data-astro-cid-uj4q7ace=""><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="344.70000000000005" y="139" width="12.8" height="12.8" transform="rotate(45 351.1 145.4)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="351.1" y="169.4" data-astro-cid-uj4q7ace="">1</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="448.90000000000003" y="124.6" width="12.8" height="12.8" transform="rotate(45 455.3 131)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="455.3" y="155" data-astro-cid-uj4q7ace="">4</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="501" y="122.19999999999999" width="12.8" height="12.8" transform="rotate(45 507.4 128.6)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="507.4" y="152.6" data-astro-cid-uj4q7ace="">8</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="553" y="119.89999999999999" width="12.8" height="12.8" transform="rotate(45 559.4 126.3)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="559.4" y="150.3" data-astro-cid-uj4q7ace="">16</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours" x="605.1" y="117.5" width="12.8" height="12.8" transform="rotate(45 611.5 123.9)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="611.5" y="147.9" data-astro-cid-uj4q7ace="">32</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-ours-light" x="325.20000000000005" y="172.7" width="12.8" height="12.8" transform="rotate(45 331.6 179.1)" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="343.6" y="183.1" data-astro-cid-uj4q7ace="">verified data only</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><circle class="chartscatter-ours-light" cx="357.3" cy="349.6" r="7" data-astro-cid-uj4q7ace="true"></circle></g><text class="chartscatter-point-label chartscatter-anchor-end" x="333.3" y="353.6" data-astro-cid-uj4q7ace="">original BIRD Train + reward shaping</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="380.4" y="240.4" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-end" x="374.4" y="236.4" data-astro-cid-uj4q7ace="">OpenSearch (GPT-5.2)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="389.3" y="254.7" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="407.3" y="274.7" data-astro-cid-uj4q7ace="">GenaSQL (GPT-5.2)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="524.3" y="334.1" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="542.3" y="344.1" data-astro-cid-uj4q7ace="">Contextual (XiYan-32B)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="158" y="382.2" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label chartscatter-anchor-middle" x="164" y="412.2" data-astro-cid-uj4q7ace="">CSC-SQL (XiYan-32B)</text></g><g class="chartscatter-point" data-astro-cid-uj4q7ace=""><g class="chartscatter-mark" data-astro-cid-uj4q7ace=""><rect class="chartscatter-scaffold" x="207.4" y="379.6" width="12" height="12" data-astro-cid-uj4q7ace="true"></rect></g><text class="chartscatter-point-label" x="225.4" y="377.6" data-astro-cid-uj4q7ace="">SHARE (GPT-5.2)</text></g></g></svg></div><figcaption>ReViSQL-K2.6 on Arcwise-Plat-SQL with 1, 4, 8, 16, and 32 samples against five open-source pipelines. Baselines are built on GPT-5.2 (GenaSQL, OpenSearch, SHARE) or XiYanSQL-QwenCoder-32B-2412 (CSC-SQL, Contextual). The 92.96% human level is marked with a line. Our model exceeds every pipeline by 8 to 22 points at comparable or lower cost per query.</figcaption></figure> <h2 id="conclusion">Conclusion</h2> <p>When AI shows poor performance in a domain-specific task, the common response is to add scaffolding around the task execution. This approach improved performance for text-to-SQL models somewhat, but it ultimately hits the ceiling imposed by the base model’s capability. This doesn’t mean that scaffolding is worthless. Rather, it suggests that the task knowledge contained in the scaffold belongs in the training signal, the same training that would uplift the model’s overall capacity.</p> <p>The common thread with our previous work on <a href="https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/">AI trained for financial judgment</a> is that custom models can outperform the frontier on a wide range of tasks involving expert taste and judgment, often at a fraction of the cost. This requires expert judgment to be part of the training process: in identifying how AI fails, labeling the training data, and aligning training to the intended behavior.</p> <p>In the case of text-to-SQL we saw major improvements from both a rigorous cleanup of the data and the shaping of the reward function to teach the model the correct skill. This took more effort upfront, but the resulting model achieves better performance at lower cost than both humans and scaffolded models. Putting task expertise into task-specific training is ultimately what scales.</p> <h2 id="citation">Citation</h2> <p>Please cite this work as:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2);overflow-x:auto" tabindex="0" data-language="plaintext"><code><span class="line"><span>Zhu, Yuxuan et al., "Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering", arXiv:2603.20004 (2026).</span></span></code></pre> <p>BibTeX:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2);overflow-x:auto" tabindex="0" data-language="plaintext"><code><span class="line"><span>@article{zhu2026revisql,</span></span> <span class="line"><span> title = {Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering},</span></span> <span class="line"><span> author = {Zhu, Yuxuan and Jin, Tengjun and Choi, Yoojin and Kang, Daniel},</span></span> <span class="line"><span> journal = {arXiv preprint arXiv:2603.20004},</span></span> <span class="line"><span> year = {2026}</span></span> <span class="line"><span>}</span></span></code></pre></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/putting-task-expertise-into-rl/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Announcing Safety Research Grants

<div class="post-cover"> <div class="grant-research-directions" data-grant-research-directions="" role="img" aria-labelledby="grant-research-directions-description"> <span id="grant-research-directions-description" class="grant-research-directions-description">Seven labeled technical figures: defense and offense with a measured gap; red hazardous particles mostly filtered while hollow benign particles are mostly retained; robust and fragile marks under strike; a trained cell producing emergent ones; a training trajectory forking between the true reward R* and a proxy optimum; a gain region between a ceiling and baseline; and measured points with a projection and break.</span> <svg width="0" height="0" aria-hidden="true" focusable="false" style="position:absolute;overflow:hidden"><defs><pattern id="grant-generalization-hatch" patternUnits="userSpaceOnUse" width="4" height="4"><line x1="-1" y1="5" x2="5" y2="-1" stroke="#9A3973" stroke-width="0.45" opacity="0.58"></line></pattern><pattern id="grant-trained-hatch" patternUnits="userSpaceOnUse" width="4" height="4"><line x1="-1" y1="5" x2="5" y2="-1" stroke="#716E68" stroke-width="0.45" opacity="0.58"></line></pattern><pattern id="grant-proxy-hatch" patternUnits="userSpaceOnUse" width="4" height="4"><line x1="-1" y1="5" x2="5" y2="-1" stroke="#0E9972" stroke-width="0.45" opacity="0.58"></line></pattern></defs></svg> <svg width="0" height="0" aria-hidden="true" focusable="false" style="position:absolute;overflow:hidden"><defs><clipPath id="grant-frontier-bars-clip" clipPathUnits="userSpaceOnUse"><path d="M106.00,384.53L108.13,381.45L110.27,378.50L112.40,375.68L114.54,372.98L116.67,370.38L118.80,367.90L120.94,365.51L123.07,363.23L125.20,361.04L127.34,358.95L129.47,356.94L131.61,355.01L133.74,353.17L135.87,351.40L138.01,349.71L140.14,348.08L142.28,346.52L144.41,345.03L146.54,343.60L148.68,342.23L150.81,340.92L152.95,339.66L155.08,338.46L157.21,337.30L159.35,336.19L161.48,335.13L163.61,334.11L165.75,333.14L167.88,332.20L170.02,331.31L172.15,330.45L174.28,329.63L176.42,328.84L178.55,328.08L180.69,327.36L182.82,326.67L184.95,326.00L187.09,325.36L189.22,324.75L191.36,324.17L193.49,323.61L195.62,323.07L197.76,322.56L199.89,322.06L202.02,321.59L204.16,321.14L206.29,320.70L208.43,320.28L210.56,319.89L212.69,319.50L214.83,319.14L216.96,318.79L219.10,318.45L221.23,318.13L223.36,317.82L225.50,317.52L227.63,317.24L229.76,316.96L231.90,316.70L234.03,316.45L236.17,316.21L238.30,315.98L240.43,315.76L242.57,315.55L244.70,315.35L246.84,315.16L248.97,314.97L251.10,314.79L253.24,314.62L255.37,314.46L257.51,314.30L259.64,314.15L259.64,384.53Z" transform="translate(0 1.4)"></path></clipPath><filter id="grant-frontier-dense-lines" filterUnits="userSpaceOnUse" x="104" y="285" width="166" height="132" color-interpolation-filters="sRGB"><feOffset in="SourceGraphic" dx="-3.135" dy="0" result="offset"></feOffset><feMerge><feMergeNode in="offset"></feMergeNode><feMergeNode in="SourceGraphic"></feMergeNode></feMerge></filter></defs></svg> <div class="grant-research-directions-static" aria-hidden="true"> <svg class="grant-research-directions-panel" data-grant-static-panel="basin" viewBox="89 71 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-basin"><line x1="106" y1="215.5" x2="268" y2="215.5" stroke="#C3C0B9" stroke-width="0.70"></line><path d="M106.00,195.94L108.03,194.13L110.05,192.36L112.08,190.64L114.10,188.97L116.13,187.34L118.15,185.75L120.17,184.22L122.20,182.73L124.22,181.28L126.25,179.88L128.28,178.53L130.30,177.22L132.32,175.96L134.35,174.75L136.38,173.58L138.40,172.45L140.43,171.37L142.45,170.34L144.47,169.36L146.50,168.42L148.53,167.52L150.55,166.67L152.57,165.87L154.60,165.11L156.63,164.40L158.65,163.74L160.68,163.12L162.70,162.54L164.72,162.02L166.75,161.54L168.78,161.10L170.80,160.71L172.82,160.37L174.85,160.07L176.88,159.82L178.90,159.61L180.93,159.45L182.95,159.33L184.97,159.26L187.00,159.24L189.03,159.26L191.05,159.33L193.07,159.45L195.10,159.61L197.13,159.82L199.15,160.07L201.18,160.37L203.20,160.71L205.22,161.10L207.25,161.54L209.28,162.02L211.30,162.54L213.32,163.12L215.35,163.74L217.38,164.40L219.40,165.11L221.43,165.87L223.45,166.67L225.47,167.52L227.50,168.42L229.53,169.36L231.55,170.34L233.57,171.37L235.60,172.45L237.63,173.58L239.65,174.75L241.68,175.96L243.70,177.22L245.72,178.53L247.75,179.88L249.78,181.28L251.80,182.73L253.82,184.22L255.85,185.75L257.88,187.34L259.90,188.97L261.93,190.64L263.95,192.36L265.98,194.13L268.00,195.94" fill="none" stroke="#8F8C85" stroke-width="0.70" stroke-linejoin="round" stroke-dasharray="3 3"></path><path d="M106.00,195.68L108.03,191.85L110.05,188.12L112.08,184.48L114.10,180.95L116.13,177.50L118.15,174.16L120.17,170.91L122.20,167.76L124.22,164.71L126.25,161.75L128.28,158.89L130.30,156.13L132.32,153.46L134.35,150.89L136.38,148.42L138.40,146.05L140.43,143.77L142.45,141.59L144.47,139.50L146.50,137.52L148.53,135.63L150.55,133.83L152.57,132.14L154.60,130.54L156.63,129.03L158.65,127.63L160.68,126.32L162.70,125.11L164.72,123.99L166.75,122.98L168.78,122.06L170.80,121.23L172.82,120.50L174.85,119.87L176.88,119.34L178.90,118.90L180.93,118.57L182.95,118.32L184.97,118.18L187.00,118.13L189.03,118.18L191.05,118.32L193.07,118.57L195.10,118.90L197.13,119.34L199.15,119.87L201.18,120.50L203.20,121.23L205.22,122.06L207.25,122.98L209.28,123.99L211.30,125.11L213.32,126.32L215.35,127.63L217.38,129.03L219.40,130.54L221.43,132.14L223.45,133.83L225.47,135.63L227.50,137.52L229.53,139.50L231.55,141.59L233.57,143.77L235.60,146.05L237.63,148.42L239.65,150.89L241.68,153.46L243.70,156.13L245.72,158.89L247.75,161.75L249.78,164.71L251.80,167.76L253.82,170.91L255.85,174.16L257.88,177.50L259.90,180.95L261.93,184.48L263.95,188.12L265.98,191.85L268.00,195.68" fill="none" stroke="#0155BF" stroke-width="1.00" stroke-linejoin="round"></path><circle cx="188.04" cy="159.25" r="3.04" fill="none" stroke="#8F8C85" stroke-width="0.70" style="opacity: 1"></circle><circle cx="207.23" cy="122.97" r="3.44" fill="#0155BF" stroke="#0155BF" stroke-width="0.86"></circle><text x="108" y="124" class="grant-research-directions-label">ROBUST</text><text x="108" y="166" class="grant-research-directions-label">FRAGILE</text><g><line x1="138.4" y1="90" x2="201.37" y2="120.16" stroke="#0155BF" stroke-width="0.70"></line><line x1="201.37" y1="120.16" x2="195.37" y2="119.88" stroke="#0155BF" stroke-width="0.70"></line><line x1="201.37" y1="120.16" x2="197.39" y2="115.66" stroke="#0155BF" stroke-width="0.70"></line></g><g><line x1="235.6" y1="90" x2="211.47" y2="118.04" stroke="#0155BF" stroke-width="0.70"></line><line x1="211.47" y1="118.04" x2="213.30" y2="112.33" stroke="#0155BF" stroke-width="0.70"></line><line x1="211.47" y1="118.04" x2="216.84" y2="115.37" stroke="#0155BF" stroke-width="0.70"></line></g></g></svg> <svg class="grant-research-directions-panel" data-grant-static-panel="filter" viewBox="321 71 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-filter"><line x1="338" y1="158.082" x2="405.5" y2="158.082" stroke="#EF402C" stroke-width="1.00"></line><line x1="432.5" y1="158.082" x2="500" y2="158.082" stroke="#EF402C" stroke-width="1.00"></line><line x1="405.5" y1="158.083" x2="405.5" y2="148.484" stroke="#EF402C" stroke-width="0.70"></line><line x1="432.5" y1="158.083" x2="432.5" y2="148.484" stroke="#EF402C" stroke-width="0.70"></line><line x1="419" y1="152.322" x2="419" y2="167.68" stroke="#8F8C85" stroke-width="0.58" stroke-dasharray="2 3"></line><text x="338" y="145" class="grant-research-directions-label">RETAINED</text><text x="338" y="214" class="grant-research-directions-label">FILTERED</text><g><circle cx="410.48" cy="210.00" r="2.4" fill="none" stroke="#8F8C85" stroke-width="0.70" style="opacity: 1"></circle><circle cx="458.23" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="445.84" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="359.76" cy="150.28" r="2.6" fill="#EF402C" stroke="none" style="opacity: 1"></circle><circle cx="414.47" cy="210.00" r="2.4" fill="none" stroke="#8F8C85" stroke-width="0.70" style="opacity: 0.972"></circle><circle cx="358.05" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 0.829"></circle><circle cx="384.81" cy="150.28" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 0.617"></circle><circle cx="483.53" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 0.435"></circle><circle cx="418.42" cy="210.00" r="2.4" fill="none" stroke="#8F8C85" stroke-width="0.70" style="opacity: 0.208"></circle><circle cx="351.52" cy="150.28" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 0.041"></circle><circle cx="383.46" cy="91.44" r="2.6" fill="#EF402C" stroke="none" style="opacity: 0.023"></circle><circle cx="457.22" cy="87.37" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 0.616"></circle><circle cx="418.63" cy="98.44" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="365.73" cy="86.14" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="365.53" cy="108.27" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="476.53" cy="103.91" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="423.94" cy="108.39" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="363.35" cy="103.18" r="2.6" fill="#EF402C" stroke="none" style="opacity: 1"></circle><circle cx="444.88" cy="102.46" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="379.28" cy="113.05" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="426.29" cy="124.57" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="455.40" cy="117.05" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="354.55" cy="128.99" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="446.72" cy="129.03" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="418.41" cy="142.57" r="2.6" fill="#EF402C" stroke="none" style="opacity: 1"></circle><circle cx="383.48" cy="134.12" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="398.83" cy="143.89" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="382.81" cy="145.18" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="417.10" cy="160.35" r="2.4" fill="none" stroke="#8F8C85" stroke-width="0.70" style="opacity: 1"></circle><circle cx="474.47" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="389.20" cy="150.28" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="378.36" cy="154.68" r="2.6" fill="#EF402C" stroke="none" style="opacity: 1"></circle><circle cx="423.32" cy="198.28" r="2.4" fill="none" stroke="#8F8C85" stroke-width="0.70" style="opacity: 1"></circle><circle cx="479.12" cy="150.28" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="400.30" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle><circle cx="357.31" cy="154.68" r="2.4" fill="none" stroke="#C3C0B9" stroke-width="0.70" style="opacity: 1"></circle></g></g></svg> <svg class="grant-research-directions-panel" data-grant-static-panel="gap" viewBox="553 71 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-gap"><line x1="573.076" y1="215.5" x2="732" y2="215.5" stroke="#C3C0B9" stroke-width="0.70"></line><line x1="573.076" y1="90" x2="573.076" y2="215.5" stroke="#C3C0B9" stroke-width="0.70"></line><defs><clipPath id="grant-gap-hatch-static" clipPathUnits="userSpaceOnUse"><path d="M573.08,204.49L575.14,201.47L577.21,199.23L579.27,197.22L581.34,195.33L583.40,193.54L585.47,191.82L587.53,190.16L589.60,188.55L591.66,186.97L593.72,185.43L595.79,183.92L597.85,182.44L599.92,180.98L601.98,179.54L604.05,178.13L606.11,176.73L608.18,175.35L610.24,173.99L612.31,172.64L614.37,171.30L616.44,169.98L618.50,168.67L620.57,167.38L622.63,166.09L624.70,164.82L626.76,163.55L628.83,162.30L630.89,161.05L632.96,159.82L635.02,158.59L637.09,157.37L639.15,156.16L641.22,154.95L643.28,153.75L645.35,152.56L647.41,151.38L649.48,150.20L651.54,149.03L653.61,147.87L655.67,146.71L657.74,145.56L659.80,144.41L661.87,143.27L663.93,142.13L666.00,141.00L668.06,139.87L670.13,138.75L672.19,137.63L674.25,136.52L676.32,135.42L678.38,134.31L680.45,133.21L682.51,132.12L684.58,131.03L686.64,129.94L688.71,128.86L690.77,127.78L692.84,126.71L694.90,125.64L696.97,124.57L699.03,123.50L701.10,122.44L703.16,121.39L705.23,120.33L707.29,119.28L709.36,118.24L711.42,117.19L713.49,116.15L715.55,115.11L717.62,114.08L719.68,113.05L721.75,112.02L721.75,156.05L719.68,156.69L717.62,157.33L715.55,157.97L713.49,158.61L711.42,159.25L709.36,159.90L707.29,160.54L705.23,161.18L703.16,161.82L701.10,162.47L699.03,163.11L696.97,163.76L694.90,164.40L692.84,165.05L690.77,165.69L688.71,166.34L686.64,166.99L684.58,167.64L682.51,168.28L680.45,168.93L678.38,169.58L676.32,170.23L674.25,170.89L672.19,171.54L670.13,172.19L668.06,172.84L666.00,173.50L663.93,174.15L661.87,174.81L659.80,175.46L657.74,176.12L655.67,176.78L653.61,177.44L651.54,178.10L649.48,178.76L647.41,179.42L645.35,180.08L643.28,180.74L641.22,181.41L639.15,182.07L637.09,182.74L635.02,183.41L632.96,184.07L630.89,184.74L628.83,185.41L626.76,186.09L624.70,186.76L622.63,187.43L620.57,188.11L618.50,188.79L616.44,189.47L614.37,190.15L612.31,190.83L610.24,191.51L608.18,192.20L606.11,192.89L604.05,193.58L601.98,194.27L599.92,194.96L597.85,195.66L595.79,196.36L593.72,197.07L591.66,197.77L589.60,198.48L587.53,199.20L585.47,199.92L583.40,200.65L581.34,201.38L579.27,202.13L577.21,202.88L575.14,203.66L573.08,204.49Z"></path></clipPath></defs><g clip-path="url(#grant-gap-hatch-static)" opacity="0.48"><line x1="448" y1="220" x2="598" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="452" y1="220" x2="602" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="456" y1="220" x2="606" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="460" y1="220" x2="610" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="464" y1="220" x2="614" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="468" y1="220" x2="618" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="472" y1="220" x2="622" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="476" y1="220" x2="626" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="480" y1="220" x2="630" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="484" y1="220" x2="634" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="488" y1="220" x2="638" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="492" y1="220" x2="642" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="496" y1="220" x2="646" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="500" y1="220" x2="650" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="504" y1="220" x2="654" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="508" y1="220" x2="658" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="512" y1="220" x2="662" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="516" y1="220" x2="666" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="520" y1="220" x2="670" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="524" y1="220" x2="674" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="528" y1="220" x2="678" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="532" y1="220" x2="682" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="536" y1="220" x2="686" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="540" y1="220" x2="690" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="544" y1="220" x2="694" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="548" y1="220" x2="698" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="552" y1="220" x2="702" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="556" y1="220" x2="706" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="560" y1="220" x2="710" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="564" y1="220" x2="714" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="568" y1="220" x2="718" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="572" y1="220" x2="722" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="576" y1="220" x2="726" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="580" y1="220" x2="730" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="584" y1="220" x2="734" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="588" y1="220" x2="738" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="592" y1="220" x2="742" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="596" y1="220" x2="746" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="600" y1="220" x2="750" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="604" y1="220" x2="754" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="608" y1="220" x2="758" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="612" y1="220" x2="762" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="616" y1="220" x2="766" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="620" y1="220" x2="770" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="624" y1="220" x2="774" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="628" y1="220" x2="778" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="632" y1="220" x2="782" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="636" y1="220" x2="786" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="640" y1="220" x2="790" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="644" y1="220" x2="794" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="648" y1="220" x2="798" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="652" y1="220" x2="802" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="656" y1="220" x2="806" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="660" y1="220" x2="810" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="664" y1="220" x2="814" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="668" y1="220" x2="818" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="672" y1="220" x2="822" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="676" y1="220" x2="826" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="680" y1="220" x2="830" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="684" y1="220" x2="834" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="688" y1="220" x2="838" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="692" y1="220" x2="842" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="696" y1="220" x2="846" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="700" y1="220" x2="850" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="704" y1="220" x2="854" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="708" y1="220" x2="858" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="712" y1="220" x2="862" y2="70" stroke="var(--blue)" stroke-width="0.45"></line><line x1="716" y1="220" x2="866" y2="70" stroke="var(--blue)" stroke-width="0.45"></line></g><path d="M573.08,204.49L575.14,201.47L577.21,199.23L579.27,197.22L581.34,195.33L583.40,193.54L585.47,191.82L587.53,190.16L589.60,188.55L591.66,186.97L593.72,185.43L595.79,183.92L597.85,182.44L599.92,180.98L601.98,179.54L604.05,178.13L606.11,176.73L608.18,175.35L610.24,173.99L612.31,172.64L614.37,171.30L616.44,169.98L618.50,168.67L620.57,167.38L622.63,166.09L624.70,164.82L626.76,163.55L628.83,162.30L630.89,161.05L632.96,159.82L635.02,158.59L637.09,157.37L639.15,156.16L641.22,154.95L643.28,153.75L645.35,152.56L647.41,151.38L649.48,150.20L651.54,149.03L653.61,147.87L655.67,146.71L657.74,145.56L659.80,144.41L661.87,143.27L663.93,142.13L666.00,141.00L668.06,139.87L670.13,138.75L672.19,137.63L674.25,136.52L676.32,135.42L678.38,134.31L680.45,133.21L682.51,132.12L684.58,131.03L686.64,129.94L688.71,128.86L690.77,127.78L692.84,126.71L694.90,125.64L696.97,124.57L699.03,123.50L701.10,122.44L703.16,121.39L705.23,120.33L707.29,119.28L709.36,118.24L711.42,117.19L713.49,116.15L715.55,115.11L717.62,114.08L719.68,113.05L721.75,112.02" fill="none" stroke="var(--blue)" stroke-width="1.00" stroke-linejoin="round"></path><path d="M573.08,204.49L575.14,203.66L577.21,202.88L579.27,202.13L581.34,201.38L583.40,200.65L585.47,199.92L587.53,199.20L589.60,198.48L591.66,197.77L593.72,197.07L595.79,196.36L597.85,195.66L599.92,194.96L601.98,194.27L604.05,193.58L606.11,192.89L608.18,192.20L610.24,191.51L612.31,190.83L614.37,190.15L616.44,189.47L618.50,188.79L620.57,188.11L622.63,187.43L624.70,186.76L626.76,186.09L628.83,185.41L630.89,184.74L632.96,184.07L635.02,183.41L637.09,182.74L639.15,182.07L641.22,181.41L643.28,180.74L645.35,180.08L647.41,179.42L649.48,178.76L651.54,178.10L653.61,177.44L655.67,176.78L657.74,176.12L659.80,175.46L661.87,174.81L663.93,174.15L666.00,173.50L668.06,172.84L670.13,172.19L672.19,171.54L674.25,170.89L676.32,170.23L678.38,169.58L680.45,168.93L682.51,168.28L684.58,167.64L686.64,166.99L688.71,166.34L690.77,165.69L692.84,165.05L694.90,164.40L696.97,163.76L699.03,163.11L701.10,162.47L703.16,161.82L705.23,161.18L707.29,160.54L709.36,159.90L711.42,159.25L713.49,158.61L715.55,157.97L717.62,157.33L719.68,156.69L721.75,156.05" fill="none" stroke="#8F8C85" stroke-width="0.70" stroke-linejoin="round"></path><circle cx="573.076" cy="204.491" r="3.08" fill="var(--blue)" stroke="var(--blue)" stroke-width="0.86"></circle><text x="592" y="116" class="grant-research-directions-label">DEFENSE</text><text x="592" y="209" class="grant-research-directions-label">OFFENSE</text><text x="748.00" y="138.04" class="grant-research-directions-label" text-anchor="end" style="opacity: 1">Δ</text></g></svg> <svg class="grant-research-directions-panel" data-grant-static-panel="frontier" viewBox="89 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-frontier"><line x1="106" y1="413" x2="268" y2="413" stroke="#C3C0B9" stroke-width="0.70"></line><line x1="106" y1="287.5" x2="106" y2="413" stroke="#C3C0B9" stroke-width="0.70"></line><g><line x1="112.271" y1="375.85050989334934" x2="112.271" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="118.542" y1="368.1952700465783" x2="118.542" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="124.813" y1="361.4395499797746" x2="124.813" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="131.084" y1="355.4776526996699" x2="131.084" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="137.355" y1="350.21630101146457" x2="137.355" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="143.626" y1="345.5731781452996" x2="143.626" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="149.897" y1="341.47563986466474" x2="149.897" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="156.168" y1="337.85957790696443" x2="156.168" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="162.439" y1="334.66841697413673" x2="162.439" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="168.71" y1="331.8522295806879" x2="168.71" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="174.981" y1="329.3669549104467" x2="174.981" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="181.252" y1="327.1737094606193" x2="181.252" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="187.523" y1="325.2381786877887" x2="187.523" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="193.79399999999998" y1="323.5300801378206" x2="193.79399999999998" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="200.065" y1="322.0226896600468" x2="200.065" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="206.336" y1="320.69242329308236" x2="206.336" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="212.607" y1="319.5184682806491" x2="212.607" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="218.878" y1="318.4824574444419" x2="218.878" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="225.149" y1="317.56818181942106" x2="225.149" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="231.42000000000002" y1="316.7613370555502" x2="231.42000000000002" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="237.691" y1="316.04929961829384" x2="237.691" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="243.962" y1="315.42092928640704" x2="243.962" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="250.233" y1="314.8663948569859" x2="250.233" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="256.504" y1="314.3770203308364" x2="256.504" y2="384.525" stroke="#DDDAD4" stroke-width="0.58"></line></g><line x1="106" y1="384.525" x2="268" y2="384.525" stroke="#8F8C85" stroke-width="0.58" stroke-dasharray="3 3"></line><line x1="106" y1="310.701" x2="268" y2="310.701" stroke="#8F8C85" stroke-width="0.58" stroke-dasharray="3 3"></line><path d="M106.00,384.53L108.13,381.45L110.27,378.50L112.40,375.68L114.54,372.98L116.67,370.38L118.80,367.90L120.94,365.51L123.07,363.23L125.20,361.04L127.34,358.95L129.47,356.94L131.61,355.01L133.74,353.17L135.87,351.40L138.01,349.71L140.14,348.08L142.28,346.52L144.41,345.03L146.54,343.60L148.68,342.23L150.81,340.92L152.95,339.66L155.08,338.46L157.21,337.30L159.35,336.19L161.48,335.13L163.61,334.11L165.75,333.14L167.88,332.20L170.02,331.31L172.15,330.45L174.28,329.63L176.42,328.84L178.55,328.08L180.69,327.36L182.82,326.67L184.95,326.00L187.09,325.36L189.22,324.75L191.36,324.17L193.49,323.61L195.62,323.07L197.76,322.56L199.89,322.06L202.02,321.59L204.16,321.14L206.29,320.70L208.43,320.28L210.56,319.89L212.69,319.50L214.83,319.14L216.96,318.79L219.10,318.45L221.23,318.13L223.36,317.82L225.50,317.52L227.63,317.24L229.76,316.96L231.90,316.70L234.03,316.45L236.17,316.21L238.30,315.98L240.43,315.76L242.57,315.55L244.70,315.35L246.84,315.16L248.97,314.97L251.10,314.79L253.24,314.62L255.37,314.46L257.51,314.30L259.64,314.15" fill="none" stroke="#716E68" stroke-width="1.00" stroke-linejoin="round"></path><circle cx="259.639" cy="314.1543733288897" r="3.34" fill="#716E68" stroke="#716E68" stroke-width="0.86"></circle><text x="108" y="306" class="grant-research-directions-label">CEILING</text><text x="108" y="398" class="grant-research-directions-label">BASELINE</text><text x="268" y="352" class="grant-research-directions-label" text-anchor="end">GAIN</text></g></svg> <svg class="grant-research-directions-panel" data-grant-static-panel="extrapolation" viewBox="321 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-extrapolation"><g><line x1="338" y1="287.5" x2="497.923" y2="287.5" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="287.5" x2="338" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="308.416" x2="497.923" y2="308.416" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="365" y1="287.5" x2="365" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="329.332" x2="497.923" y2="329.332" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="392" y1="287.5" x2="392" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="350.248" x2="497.923" y2="350.248" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="419" y1="287.5" x2="419" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="371.164" x2="497.923" y2="371.164" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="446" y1="287.5" x2="446" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="392.08" x2="497.923" y2="392.08" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="473" y1="287.5" x2="473" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="412.996" x2="497.923" y2="412.996" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="500" y1="287.5" x2="500" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="338" y1="413" x2="497.923" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="500" y1="287.5" x2="500" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line></g><line x1="338" y1="413" x2="497.923" y2="413" stroke="#C3C0B9" stroke-width="0.70"></line><line x1="338" y1="287.5" x2="338" y2="413" stroke="#C3C0B9" stroke-width="0.70"></line><line x1="342.154" y1="393.925" x2="410.40" y2="355.43" stroke="#716E68" stroke-width="1.00"></line><line x1="410.40" y1="355.43" x2="493.769" y2="308.416" stroke="#8F8C85" stroke-width="0.70" stroke-dasharray="4 4"></line><g style="opacity: 1"><path d="M446,335.399 C462.615,329.543 473,340.419 489.615,349.204" fill="none" stroke="#716E68" stroke-width="1.00" stroke-linejoin="round"></path><circle cx="446" cy="335.357" r="4.15" fill="#FFFFFF" stroke="#716E68" stroke-width="0.86"></circle><text x="455" y="326" class="grant-research-directions-label">BREAK</text></g><circle cx="346.308" cy="389.636" r="3.11" fill="#716E68" stroke="#716E68" stroke-width="0.86" style="opacity: 1"></circle><circle cx="365" cy="383.319" r="3.11" fill="#716E68" stroke="#716E68" stroke-width="0.86" style="opacity: 1"></circle><circle cx="383.692" cy="372.296" r="3.11" fill="#716E68" stroke="#716E68" stroke-width="0.86" style="opacity: 1"></circle><circle cx="402.385" cy="360.917" r="3.11" fill="#716E68" stroke="#716E68" stroke-width="0.86" style="opacity: 1"></circle><text x="340" y="406" class="grant-research-directions-label">MEASURED</text><text x="500" y="300" class="grant-research-directions-label" text-anchor="end">PROJECTED</text></g></svg> <svg class="grant-research-directions-panel" data-grant-static-panel="proxy" viewBox="553 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-proxy"><ellipse cx="612" cy="316" rx="12" ry="8" transform="rotate(-14 612 316)" fill="none" stroke="#DDDAD4" stroke-width="0.58"></ellipse><ellipse cx="612" cy="316" rx="24" ry="16" transform="rotate(-14 612 316)" fill="none" stroke="#DDDAD4" stroke-width="0.58"></ellipse><ellipse cx="612" cy="316" rx="36" ry="24" transform="rotate(-14 612 316)" fill="none" stroke="#DDDAD4" stroke-width="0.58"></ellipse><ellipse cx="686" cy="380" rx="11" ry="7" transform="rotate(-14 686 380)" fill="none" stroke="#C3C0B9" stroke-width="0.58"></ellipse><ellipse cx="686" cy="380" rx="22" ry="14" transform="rotate(-14 686 380)" fill="none" stroke="#C3C0B9" stroke-width="0.58"></ellipse><ellipse cx="686" cy="380" rx="33" ry="21" transform="rotate(-14 686 380)" fill="none" stroke="#C3C0B9" stroke-width="0.58"></ellipse><path d="M576,360 C586,357 596,354 604,352 C606,339 609,325 612,316" fill="none" stroke="#8F8C85" stroke-width="0.70" stroke-linejoin="round"></path><path d="M576,360 C586,357 596,354 604,352 C624,350 654,364 686,380" fill="none" stroke="#716E68" stroke-width="0.90" stroke-linejoin="round"></path><circle cx="588" cy="356" r="2.4" fill="#FFFFFF" stroke="#8F8C85" stroke-width="0.86"></circle><circle cx="604" cy="352" r="2.4" fill="#FFFFFF" stroke="#8F8C85" stroke-width="0.86"></circle><circle cx="609" cy="333" r="2.4" fill="#FFFFFF" stroke="#8F8C85" stroke-width="0.86"></circle><circle cx="632" cy="354" r="2.4" fill="#FFFFFF" stroke="#716E68" stroke-width="0.86"></circle><circle cx="659" cy="367" r="2.4" fill="#FFFFFF" stroke="#716E68" stroke-width="0.86"></circle><circle cx="612" cy="316" r="3" fill="#FFFFFF" stroke="#8F8C85" stroke-width="0.70"></circle><circle cx="686" cy="380" r="3.4" fill="#716E68" stroke="#716E68" stroke-width="0.86"></circle><text x="570" y="296" class="grant-research-directions-label">R*</text><text x="732" y="404" class="grant-research-directions-label" text-anchor="end">PROXY</text></g></svg> <svg class="grant-research-directions-panel" data-grant-static-panel="generalization" viewBox="553 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"><g id="grant-direction-generalization"><g><line x1="570" y1="287.5" x2="732" y2="287.5" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="287.5" x2="570" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="305.429" x2="732" y2="305.429" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="593.143" y1="287.5" x2="593.143" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="323.358" x2="732" y2="323.358" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="616.2860000000001" y1="287.5" x2="616.2860000000001" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="341.287" x2="732" y2="341.287" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="639.429" y1="287.5" x2="639.429" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="359.216" x2="732" y2="359.216" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="662.572" y1="287.5" x2="662.572" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="377.145" x2="732" y2="377.145" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="685.715" y1="287.5" x2="685.715" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="395.07399999999996" x2="732" y2="395.07399999999996" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="708.858" y1="287.5" x2="708.858" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="570" y1="413.003" x2="732" y2="413.003" stroke="#DDDAD4" stroke-width="0.58"></line><line x1="732.001" y1="287.5" x2="732.001" y2="413" stroke="#DDDAD4" stroke-width="0.58"></line></g><text x="620" y="390" class="grant-research-directions-label">TRAINED</text><text x="732" y="300" class="grant-research-directions-label" text-anchor="end">EMERGENT</text><g style="opacity: 1"><rect width="23.143" height="17.929" fill="none" stroke="#716E68" stroke-width="0.90" x="616.2860000000001" y="287.5"></rect><path d="M604.714,386.107 Q632.3,325.3 627.9,296.5" fill="none" stroke="#8F8C85" stroke-width="0.58" stroke-linejoin="round" stroke-dasharray="3 3"></path></g><g style="opacity: 1"><rect width="23.143" height="17.929" fill="none" stroke="#716E68" stroke-width="0.90" x="708.858" y="305.429"></rect><path d="M604.714,386.107 Q678.6,334.2 720.4,314.4" fill="none" stroke="#8F8C85" stroke-width="0.58" stroke-linejoin="round" stroke-dasharray="3 3"></path></g><g style="opacity: 1"><rect width="23.143" height="17.929" fill="none" stroke="#716E68" stroke-width="0.90" x="662.572" y="323.358"></rect><path d="M604.714,386.107 Q655.4,343.2 674.1,332.3" fill="none" stroke="#8F8C85" stroke-width="0.58" stroke-linejoin="round" stroke-dasharray="3 3"></path></g><rect x="593.143" y="377.143" width="23.143" height="17.929" fill="url(#grant-trained-hatch)" stroke="#716E68" stroke-width="0.90"></rect></g></svg> </div> <div class="grant-research-directions-live" aria-hidden="true"> <svg class="grant-research-directions-panel" data-grant-panel="basin" viewBox="89 71 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> <svg class="grant-research-directions-panel" data-grant-panel="filter" viewBox="321 71 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> <svg class="grant-research-directions-panel" data-grant-panel="gap" viewBox="553 71 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> <svg class="grant-research-directions-panel" data-grant-panel="frontier" viewBox="89 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> <svg class="grant-research-directions-panel" data-grant-panel="extrapolation" viewBox="321 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> <svg class="grant-research-directions-panel" data-grant-panel="proxy" viewBox="553 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> <svg class="grant-research-directions-panel" data-grant-panel="generalization" viewBox="553 269 196 162" xmlns="http://www.w3.org/2000/svg" focusable="false"></svg> </div> </div> </div><div class="post-content-shell"><article class="content"><p>We recently released <a href="https://thinkingmachines.ai/news/introducing-inkling/">Inkling</a> and <a href="https://thinkingmachines.ai/news/inkling-small/">Inkling-Small</a> with open weights, and explained <a href="https://thinkingmachines.ai/blog/a-safe-path-to-open-weights/">how we are thinking about safety and openness</a>. Today, we are launching Tinker grants of up to $50,000 in credits for safety research on open-weight models. Below are some directions we find promising. The list is non-exhaustive on purpose. If you’re working on a safety project that could be accelerated by additional Tinker credits, we want to hear from you.</p> <h2 id="making-models-safer-to-open">Making models safer to open</h2> <p><strong>Differential acceleration of defensive over offensive capability.</strong> Many capabilities are dual-use: the same cyber skills that let a model find and patch vulnerabilities in defenders’ hands can accelerate exploitation of unpatched systems in attackers’ hands. Similar tradeoffs arise in domains like chemistry and biology. However, we hypothesize that not every skill contributes equally to both sides — some capabilities (e.g. triage, detection, hardening) may be disproportionately useful to defenders, while others (e.g. exploitation, evasion) skew toward attackers.<label for="sn-grants-bountybench" class="sidenote-number"></label><input type="checkbox" id="sn-grants-bountybench" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2505.15216">BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems</a> (Zhang et al, 2026)</span><span class="sidenote-citation-separator">, </span><label for="sn-grants-worms" class="sidenote-number"></label><input type="checkbox" id="sn-grants-worms" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2606.03811">AI Agents Enable Adaptive Computer Worms</a> (Guan et al, 2026)</span> Can we fine-tune models to differentially improve defensive skills while minimally improving offensive ones? A direct experiment would fine-tune a model toward defensive skills and then measure the uplift to each side — the offense–defense gap of the training run, rather than raw capability alone. A key question is whether any such gap reflects genuine asymmetry, rather than suppression that dissolves under attacker pressure or sandbagging that persists only under measurement.</p> <p><strong>Building classifiers for hazardous data at scale.</strong> Training data can contain information that materially increases hazardous capabilities, but identifying that information is difficult.<label for="sn-grants-rathi" class="sidenote-number"></label><input type="checkbox" id="sn-grants-rathi" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2601.21571">Shaping capabilities with token-level data filtering</a> (Rathi and Radford, 2026)</span> The downstream impact of any individual document or data point is hard to predict in advance,<label for="sn-grants-magic" class="sidenote-number"></label><input type="checkbox" id="sn-grants-magic" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2504.16430">Magic: Near-optimal data attribution for deep learning</a> (Ilyas and Engstrom, 2025)</span> and constructing high-quality labels often requires scarce domain expertise. Can we train filtering models that reliably identify safety-relevant data at sufficient scale and recall, without discarding large amounts of benign scientific or technical content?<label for="sn-grants-obrien" class="sidenote-number"></label><input type="checkbox" id="sn-grants-obrien" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2508.06601">Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs</a> (O’Brien et al, 2026)</span><span class="sidenote-citation-separator">, </span><label for="sn-grants-chen" class="sidenote-number"></label><input type="checkbox" id="sn-grants-chen" class="margin-toggle"><span class="sidenote"><a href="https://alignment.anthropic.com/2025/pretraining-data-filtering/">Enhancing Model Safety through Pretraining Data Filtering</a> (Chen et al, 2025)</span> And can these classifiers be made robust against paraphrasing, obfuscation, domain shift, and changing model capabilities?<label for="sn-grants-maini" class="sidenote-number"></label><input type="checkbox" id="sn-grants-maini" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2504.16980">Safety Pretraining: Toward the Next Generation of Safe AI</a> (Maini et al, 2026)</span> Ultimately, the strongest test is downstream, and directly measurable with fine-tuning access: does filtering actually reduce hazardous capability uplift, and what concerning capabilities can still be learned from data that passes the filter?</p> <p><strong>Tamper-resistant safety training.</strong> Built-in safeguards learned during model training may not remain stable under subsequent fine-tuning. Can mitigations be developed that persist under downstream modification, including adversarial fine-tuning intended to remove safeguards and ordinary continued training that may inadvertently wash them out?<label for="sn-grants-tamirisa" class="sidenote-number"></label><input type="checkbox" id="sn-grants-tamirisa" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2408.00761">Tamper-Resistant Safeguards for Open-Weight LLMs</a> (Tamirisa et al, 2025)</span><span class="sidenote-citation-separator">, </span><label for="sn-grants-casper" class="sidenote-number"></label><input type="checkbox" id="sn-grants-casper" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2608.07514">Open Technical Problems in Open-Weight AI Model Risk Management</a> (Casper et al, 2026)</span> Designing scalable adversarial training methods that improve robustness against diverse downstream modifications is one promising approach, with generalization to held-out attack strategies a key test of success.<label for="sn-grants-che" class="sidenote-number"></label><input type="checkbox" id="sn-grants-che" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2502.05209">Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities</a> (Che et al, 2025)</span> Understanding how robustness changes with scale and architecture is another important question, while negative or impossibility results can help establish the limits of what persistent safety mitigations can achieve.</p> <h2 id="understanding-alignment-failure-modes">Understanding alignment failure modes</h2> <p><strong>Safety-relevant generalization from narrow fine-tuning.</strong> Narrow fine-tuning can sometimes produce behaviors far outside of the task that was directly trained. Emergent misalignment<label for="sn-grants-betley" class="sidenote-number"></label><input type="checkbox" id="sn-grants-betley" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2502.17424">Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs</a> (Betley et al, 2025)</span> is one striking example, but we suspect it is one instance of a broader phenomenon, and we want to characterize that phenomenon directly: its boundary conditions, its scaling properties, and the mechanisms that cause it. When does narrow harmful training generalize into broad behavioral change, and does this require intentionally adversarial data, or could it plausibly arise accidentally from ordinary downstream fine-tuning? A related compositional question: can a model separately learn planning, tool use, and relevant domain knowledge, then compose those pieces into a concerning behavior that was never demonstrated end-to-end? Understanding which of these effects are quirks of specific setups and which are robust properties of fine-tuning would materially change how downstream training should be evaluated.</p> <p><strong>Reward hacking and oversight gaming.</strong> Proxy rewards can create real failure modes. Optimizing an imperfect reward can eventually favor behavior that scores well without accomplishing the intended objective. We want to understand when reward hacking emerges, how it changes with model capability and optimization pressure, and whether early signals can predict it before it becomes severe.<label for="sn-grants-cagatan" class="sidenote-number"></label><input type="checkbox" id="sn-grants-cagatan" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2606.15385">Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds</a> (Çağatan and Zhao, 2026)</span> More concerning failures may involve models learning strategies for gaming or interfering with oversight, such as chain-of-thought monitors. A key open question is whether these strategies can be detected, and whether they transfer across tasks.<label for="sn-grants-hadida" class="sidenote-number"></label><input type="checkbox" id="sn-grants-hadida" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2601.23086">Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks</a> (Hadida et al, 2026)</span></p> <h2 id="measuring-and-forecasting-risk">Measuring and forecasting risk</h2> <p><strong>Worst-case and marginal risk estimation.</strong> Open-weight models let people make AI their own. A complementary safety challenge is understanding the maximum risk a model can pose after release. How can we estimate safeguard degradation in different safety-relevant domains, and as a function of a motivated attacker’s time, data, or optimization effort? A related quantity is marginal risk: what harmful capabilities does this model enable beyond what is already possible? Fine-tuning itself is a natural stress test here: evaluations built on adversarial fine-tuning can distinguish capabilities that are genuinely absent from those that are merely suppressed and easily re-elicited.<label for="sn-grants-tamperbench" class="sidenote-number"></label><input type="checkbox" id="sn-grants-tamperbench" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2602.06911">TamperBench: Systematically Stress-Testing LLM Safety under Fine-Tuning and Tampering</a> (Hossain et al, 2026)</span> Recent work on worst-case post-training provides useful starting points,<label for="sn-grants-wallace" class="sidenote-number"></label><input type="checkbox" id="sn-grants-wallace" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2508.03153">Estimating Worst-Case Frontier Risks of Open-Weight LLMs</a> (Wallace et al, 2026)</span> but leaves the shape and limits of this resource–risk frontier poorly understood.</p> <p><strong>Forecasting safety-relevant scaling trends.</strong> Many of the questions in this post become substantially more tractable if expensive experiments can be forecast from smaller runs. Can measurements at low post-training budgets predict downstream safety-relevant properties — including capability uplift, safeguard degradation, or reward hacking — as optimization and compute are scaled up? More broadly, we want to understand which safety-relevant scaling trends extrapolate reliably across model capability and training budget — and where those trends break down. Existing work on forecasting rare behaviors<label for="sn-grants-jones" class="sidenote-number"></label><input type="checkbox" id="sn-grants-jones" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2502.16797">Forecasting Rare Language Model Behaviors</a> (Jones et al, 2025)</span> and adversarial robustness<label for="sn-grants-panfilov" class="sidenote-number"></label><input type="checkbox" id="sn-grants-panfilov" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2505.20162">Capability-Based Scaling Trends for LLM-Based Red-Teaming</a> (Panfilov et al, 2026)</span> provides useful examples of the broader possibility.</p> <h2 id="the-grants">The grants</h2> <p>See <a href="https://thinkingmachines.ai/news/safety-research-grants/apply/">how to apply</a> for proposal requirements, selection criteria, timeline, and complete <a href="https://thinkingmachines.ai/legal/safety-research-grants-terms/">terms of service</a> for the grant program.</p></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/safety-research-grants/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Introducing Inkling-Small

<div class="post-cover compact"><div class="inkling inkling-cover" data-recipe="{&quot;seed&quot;:333549122,&quot;grid&quot;:24,&quot;count&quot;:3,&quot;mode&quot;:&quot;outline&quot;,&quot;strokeWidth&quot;:4,&quot;diffusion&quot;:3,&quot;round&quot;:0.8,&quot;declutter&quot;:2,&quot;gridlines&quot;:true,&quot;background&quot;:&quot;#FDF7ED&quot;}" data-interactive="true"><svg viewBox="0 0 660 440" role="img" aria-label="Generative ink-blob artwork" style="touch-action: pan-y;"><defs><mask id="ixmask-inkfb" maskUnits="userSpaceOnUse" x="-100" y="-100" width="860" height="640"><rect x="-100" y="-100" width="860" height="640" fill="#fff"></rect><path class="ink-ixmask-path" fill="#000" fill-rule="evenodd" d=""></path></mask><filter id="diffuse-inkfb" x="-25%" y="-25%" width="150%" height="150%" color-interpolation-filters="sRGB"><feTurbulence type="fractalNoise" baseFrequency="0.09" numOctaves="2" seed="11" result="dn"></feTurbulence><feDisplacementMap class="ink-diffuse-disp" in="SourceGraphic" in2="dn" scale="3" xChannelSelector="R" yChannelSelector="G"></feDisplacementMap></filter></defs><rect class="ink-bg" width="660" height="440" fill="#FDF7ED"></rect><g class="ink-lattice"><line x1="36" y1="24" x2="36" y2="416" stroke="#161311"></line><line x1="60.5" y1="24" x2="60.5" y2="416" stroke="#161311"></line><line x1="85" y1="24" x2="85" y2="416" stroke="#161311"></line><line x1="109.5" y1="24" x2="109.5" y2="416" stroke="#161311"></line><line x1="134" y1="24" x2="134" y2="416" stroke="#161311"></line><line x1="158.5" y1="24" x2="158.5" y2="416" stroke="#161311"></line><line x1="183" y1="24" x2="183" y2="416" stroke="#161311"></line><line x1="207.5" y1="24" x2="207.5" y2="416" stroke="#161311"></line><line x1="232" y1="24" x2="232" y2="416" stroke="#161311"></line><line x1="256.5" y1="24" x2="256.5" y2="416" stroke="#161311"></line><line x1="281" y1="24" x2="281" y2="416" stroke="#161311"></line><line x1="305.5" y1="24" x2="305.5" y2="416" stroke="#161311"></line><line x1="330" y1="24" x2="330" y2="416" stroke="#161311"></line><line x1="354.5" y1="24" x2="354.5" y2="416" stroke="#161311"></line><line x1="379" y1="24" x2="379" y2="416" stroke="#161311"></line><line x1="403.5" y1="24" x2="403.5" y2="416" stroke="#161311"></line><line x1="428" y1="24" x2="428" y2="416" stroke="#161311"></line><line x1="452.5" y1="24" x2="452.5" y2="416" stroke="#161311"></line><line x1="477" y1="24" x2="477" y2="416" stroke="#161311"></line><line x1="501.5" y1="24" x2="501.5" y2="416" stroke="#161311"></line><line x1="526" y1="24" x2="526" y2="416" stroke="#161311"></line><line x1="550.5" y1="24" x2="550.5" y2="416" stroke="#161311"></line><line x1="575" y1="24" x2="575" y2="416" stroke="#161311"></line><line x1="599.5" y1="24" x2="599.5" y2="416" stroke="#161311"></line><line x1="624" y1="24" x2="624" y2="416" stroke="#161311"></line><line x1="36" y1="24" x2="624" y2="24" stroke="#161311"></line><line x1="36" y1="48.5" x2="624" y2="48.5" stroke="#161311"></line><line x1="36" y1="73" x2="624" y2="73" stroke="#161311"></line><line x1="36" y1="97.5" x2="624" y2="97.5" stroke="#161311"></line><line x1="36" y1="122" x2="624" y2="122" stroke="#161311"></line><line x1="36" y1="146.5" x2="624" y2="146.5" stroke="#161311"></line><line x1="36" y1="171" x2="624" y2="171" stroke="#161311"></line><line x1="36" y1="195.5" x2="624" y2="195.5" stroke="#161311"></line><line x1="36" y1="220" x2="624" y2="220" stroke="#161311"></line><line x1="36" y1="244.5" x2="624" y2="244.5" stroke="#161311"></line><line x1="36" y1="269" x2="624" y2="269" stroke="#161311"></line><line x1="36" y1="293.5" x2="624" y2="293.5" stroke="#161311"></line><line x1="36" y1="318" x2="624" y2="318" stroke="#161311"></line><line x1="36" y1="342.5" x2="624" y2="342.5" stroke="#161311"></line><line x1="36" y1="367" x2="624" y2="367" stroke="#161311"></line><line x1="36" y1="391.5" x2="624" y2="391.5" stroke="#161311"></line><line x1="36" y1="416" x2="624" y2="416" stroke="#161311"></line></g><g class="ink-squares"></g><g class="ink-fx" style="filter: url(&quot;#diffuse-inkfb&quot;);"><g class="ink-marks"><path fill="none" stroke="#161311" stroke-width="4" d="M36,269.89C35.73,272.21 35.54,274.55 35.37,276.88C35.21,279.22 35.1,281.56 35.01,283.9C34.92,286.24 34.87,288.58 34.83,290.92C34.8,293.27 34.79,295.61 34.78,297.95C34.77,300.29 34.77,302.64 34.78,304.98C34.78,307.32 34.77,309.66 34.78,312C34.79,314.35 34.8,316.69 34.82,319.03C34.85,321.37 34.89,323.71 34.95,326.05C35.01,328.4 35.09,330.74 35.2,333.08C35.32,335.42 35.45,337.75 35.64,340.09C35.83,342.42 36.05,344.76 36.34,347.08C36.64,349.4 36.97,351.72 37.4,354.02C37.83,356.32 38.32,358.62 38.91,360.88C39.51,363.15 40.19,365.4 40.97,367.6C41.75,369.8 42.63,371.99 43.61,374.11C44.59,376.23 45.68,378.32 46.86,380.33C48.05,382.35 49.32,384.33 50.71,386.21C52.1,388.08 53.61,389.89 55.21,391.6C56.81,393.3 58.52,394.93 60.3,396.43C62.09,397.94 63.98,399.34 65.93,400.64C67.87,401.94 69.9,403.13 71.97,404.22C74.04,405.31 76.17,406.3 78.33,407.19C80.5,408.08 82.71,408.88 84.94,409.58C87.17,410.28 89.45,410.86 91.73,411.38C94.01,411.89 96.32,412.31 98.63,412.67C100.95,413.03 103.27,413.31 105.6,413.54C107.94,413.76 110.28,413.91 112.61,414.01C114.95,414.1 117.3,414.12 119.64,414.08C121.98,414.04 124.32,413.95 126.66,413.76C128.99,413.58 131.33,413.34 133.64,412.99C135.95,412.65 138.27,412.24 140.54,411.7C142.82,411.17 145.09,410.55 147.3,409.79C149.51,409.03 151.71,408.16 153.79,407.12C155.88,406.07 157.92,404.87 159.82,403.52C161.72,402.17 163.55,400.66 165.22,399.04C166.89,397.41 168.45,395.63 169.85,393.76C171.25,391.9 172.51,389.89 173.62,387.84C174.74,385.79 175.7,383.64 176.56,381.47C177.43,379.29 178.14,377.05 178.81,374.81C179.48,372.57 180.04,370.29 180.6,368.02C181.16,365.74 181.64,363.45 182.15,361.17C182.66,358.88 183.14,356.59 183.66,354.3C184.19,352.02 184.71,349.74 185.31,347.47C185.91,345.21 186.55,342.95 187.27,340.73C187.99,338.5 188.77,336.29 189.63,334.11C190.48,331.93 191.41,329.77 192.4,327.66C193.4,325.54 194.48,323.46 195.61,321.41C196.74,319.35 197.94,317.34 199.18,315.35C200.41,313.37 201.72,311.42 203.04,309.48C204.35,307.55 205.71,305.64 207.07,303.73C208.43,301.83 209.82,299.94 211.21,298.05C212.59,296.16 213.99,294.29 215.39,292.41C216.79,290.53 218.19,288.65 219.61,286.79C221.02,284.92 222.44,283.05 223.9,281.22C225.36,279.4 226.83,277.57 228.38,275.82C229.93,274.06 231.52,272.33 233.2,270.7C234.88,269.07 236.62,267.49 238.46,266.05C240.3,264.61 242.22,263.24 244.23,262.05C246.24,260.86 248.36,259.81 250.52,258.94C252.69,258.07 254.94,257.37 257.22,256.83C259.49,256.29 261.82,255.96 264.15,255.71C266.47,255.45 268.82,255.36 271.16,255.27C273.5,255.19 275.84,255.21 278.18,255.18C280.52,255.15 282.87,255.18 285.21,255.09C287.55,255 289.9,254.93 292.22,254.66C294.54,254.39 296.88,254.07 299.14,253.48C301.39,252.9 303.66,252.15 305.75,251.15C307.85,250.14 309.88,248.88 311.7,247.44C313.53,246.01 315.23,244.33 316.72,242.54C318.2,240.76 319.5,238.77 320.64,236.73C321.77,234.69 322.69,232.51 323.51,230.32C324.33,228.14 324.98,225.87 325.56,223.61C326.14,221.34 326.59,219.04 327,216.73C327.4,214.43 327.71,212.1 327.99,209.78C328.27,207.45 328.48,205.12 328.67,202.79C328.87,200.45 329.01,198.11 329.16,195.78C329.3,193.44 329.42,191.1 329.54,188.76C329.66,186.42 329.77,184.08 329.89,181.74C330.01,179.4 330.13,177.06 330.27,174.73C330.41,172.39 330.56,170.05 330.74,167.72C330.93,165.38 331.13,163.05 331.4,160.72C331.66,158.39 331.94,156.07 332.32,153.76C332.7,151.45 333.14,149.14 333.67,146.86C334.2,144.58 334.77,142.3 335.5,140.08C336.23,137.86 337.01,135.63 338.03,133.53C339.05,131.44 340.17,129.31 341.63,127.52C343.09,125.73 344.81,123.94 346.77,122.79C348.72,121.64 351.12,120.87 353.37,120.64C355.63,120.41 358.07,120.83 360.3,121.42C362.54,122 364.7,123.07 366.77,124.14C368.84,125.21 370.78,126.55 372.74,127.84C374.7,129.12 376.58,130.52 378.52,131.82C380.47,133.13 382.4,134.47 384.4,135.67C386.41,136.88 388.44,138.06 390.55,139.06C392.66,140.06 394.84,140.98 397.07,141.66C399.3,142.34 401.62,142.89 403.93,143.16C406.24,143.42 408.63,143.46 410.94,143.23C413.25,143.01 415.59,142.5 417.8,141.8C420.02,141.09 422.2,140.13 424.24,139.01C426.28,137.89 428.24,136.54 430.04,135.07C431.85,133.59 433.55,131.94 435.07,130.17C436.59,128.41 437.99,126.49 439.17,124.48C440.36,122.48 441.37,120.33 442.17,118.14C442.97,115.95 443.56,113.65 443.97,111.36C444.37,109.06 444.56,106.7 444.59,104.37C444.61,102.04 444.43,99.68 444.1,97.37C443.78,95.06 443.26,92.75 442.62,90.5C441.99,88.25 441.19,86.04 440.3,83.88C439.41,81.71 438.39,79.59 437.28,77.54C436.17,75.48 434.93,73.48 433.64,71.53C432.35,69.58 430.96,67.69 429.52,65.84C428.08,63.99 426.57,62.2 425.02,60.44C423.48,58.69 421.87,56.97 420.23,55.31C418.59,53.64 416.9,52.01 415.17,50.43C413.44,48.86 411.67,47.32 409.85,45.84C408.03,44.37 406.17,42.94 404.26,41.6C402.34,40.26 400.38,38.96 398.35,37.8C396.33,36.63 394.24,35.54 392.11,34.58C389.98,33.62 387.79,32.75 385.56,32.03C383.34,31.32 381.06,30.7 378.77,30.27C376.47,29.84 374.13,29.54 371.8,29.44C369.47,29.33 367.1,29.4 364.78,29.65C362.46,29.9 360.13,30.33 357.88,30.94C355.63,31.55 353.41,32.37 351.28,33.32C349.15,34.26 347.08,35.4 345.08,36.62C343.09,37.84 341.18,39.21 339.31,40.62C337.45,42.03 335.66,43.56 333.88,45.08C332.1,46.59 330.38,48.18 328.62,49.73C326.86,51.27 325.13,52.87 323.32,54.35C321.51,55.83 319.69,57.32 317.75,58.63C315.82,59.93 313.79,61.16 311.7,62.17C309.6,63.19 307.4,64.07 305.16,64.74C302.93,65.4 300.6,65.86 298.29,66.15C295.98,66.45 293.61,66.53 291.28,66.5C288.94,66.47 286.59,66.27 284.28,65.95C281.96,65.64 279.66,65.17 277.39,64.6C275.12,64.04 272.87,63.34 270.67,62.55C268.47,61.77 266.29,60.87 264.17,59.9C262.04,58.93 259.96,57.85 257.91,56.72C255.86,55.59 253.85,54.37 251.87,53.12C249.88,51.88 247.94,50.57 245.99,49.27C244.04,47.98 242.11,46.65 240.17,45.35C238.22,44.05 236.28,42.73 234.3,41.48C232.32,40.23 230.33,38.99 228.3,37.82C226.27,36.66 224.21,35.53 222.12,34.48C220.02,33.44 217.89,32.46 215.73,31.56C213.57,30.65 211.38,29.81 209.17,29.06C206.95,28.31 204.7,27.64 202.44,27.05C200.17,26.46 197.88,25.97 195.58,25.54C193.27,25.11 190.95,24.76 188.63,24.46C186.31,24.16 183.98,23.93 181.65,23.73C179.31,23.52 176.97,23.38 174.64,23.26C172.3,23.13 169.96,23.05 167.62,22.98C165.27,22.91 162.93,22.87 160.59,22.85C158.25,22.84 155.91,22.84 153.56,22.89C151.22,22.94 148.88,23.01 146.54,23.14C144.2,23.26 141.87,23.41 139.53,23.62C137.2,23.84 134.87,24.1 132.56,24.44C130.24,24.79 127.93,25.19 125.65,25.7C123.37,26.22 121.09,26.81 118.87,27.55C116.65,28.29 114.45,29.13 112.35,30.15C110.25,31.17 108.19,32.33 106.27,33.65C104.34,34.97 102.51,36.46 100.81,38.07C99.12,39.67 97.53,41.42 96.1,43.27C94.68,45.12 93.4,47.11 92.27,49.15C91.15,51.2 90.19,53.36 89.36,55.54C88.53,57.72 87.86,59.98 87.27,62.24C86.69,64.51 86.23,66.81 85.84,69.12C85.45,71.43 85.15,73.76 84.92,76.08C84.69,78.41 84.54,80.75 84.44,83.09C84.33,85.43 84.3,87.78 84.31,90.12C84.32,92.46 84.4,94.8 84.51,97.14C84.62,99.48 84.78,101.82 84.98,104.15C85.19,106.48 85.44,108.81 85.74,111.14C86.04,113.46 86.39,115.78 86.8,118.08C87.22,120.38 87.7,122.68 88.24,124.96C88.78,127.23 89.38,129.5 90.04,131.75C90.7,133.99 91.44,136.22 92.21,138.43C92.98,140.64 93.82,142.83 94.66,145.01C95.5,147.2 96.41,149.36 97.23,151.55C98.04,153.75 98.94,155.93 99.55,158.18C100.15,160.44 100.79,162.77 100.84,165.07C100.9,167.37 100.56,169.77 99.89,171.98C99.22,174.18 98.04,176.29 96.85,178.29C95.65,180.29 94.17,182.13 92.73,183.97C91.29,185.82 89.75,187.59 88.2,189.34C86.66,191.1 85.06,192.81 83.46,194.52C81.86,196.24 80.23,197.92 78.61,199.61C76.99,201.3 75.35,202.98 73.72,204.65C72.09,206.33 70.44,208 68.81,209.68C67.18,211.37 65.54,213.04 63.94,214.74C62.33,216.44 60.72,218.15 59.16,219.9C57.6,221.65 56.06,223.41 54.58,225.22C53.1,227.04 51.65,228.88 50.29,230.78C48.92,232.69 47.61,234.63 46.41,236.64C45.21,238.65 44.09,240.71 43.09,242.83C42.09,244.94 41.19,247.11 40.41,249.32C39.63,251.52 38.98,253.79 38.41,256.05C37.84,258.32 37.38,260.63 36.98,262.93C36.58,265.24 36.27,267.56 36,269.89Z"></path><path fill="none" stroke="#161311" stroke-width="4" d="M430.45,104.18C430.25,105.22 430.07,106.27 429.93,107.31C429.78,108.36 429.66,109.41 429.56,110.46C429.46,111.51 429.38,112.57 429.32,113.63C429.26,114.68 429.22,115.74 429.21,116.8C429.2,117.85 429.21,118.91 429.25,119.97C429.28,121.02 429.34,122.08 429.42,123.13C429.5,124.19 429.6,125.24 429.73,126.29C429.86,127.34 430,128.39 430.18,129.43C430.36,130.47 430.56,131.51 430.79,132.54C431.02,133.57 431.27,134.6 431.55,135.62C431.83,136.64 432.13,137.65 432.46,138.66C432.8,139.66 433.15,140.66 433.55,141.64C433.94,142.62 434.37,143.59 434.82,144.54C435.27,145.5 435.76,146.44 436.26,147.37C436.76,148.3 437.29,149.22 437.84,150.12C438.4,151.02 438.98,151.9 439.59,152.76C440.19,153.63 440.81,154.49 441.46,155.32C442.1,156.16 442.77,156.98 443.45,157.79C444.14,158.6 444.84,159.38 445.56,160.16C446.28,160.93 447.03,161.69 447.78,162.42C448.54,163.16 449.31,163.88 450.1,164.59C450.89,165.29 451.69,165.98 452.51,166.65C453.33,167.32 454.17,167.97 455.02,168.6C455.86,169.22 456.73,169.83 457.61,170.42C458.48,171.02 459.37,171.59 460.27,172.14C461.17,172.7 462.08,173.24 463,173.76C463.92,174.28 464.85,174.78 465.79,175.26C466.73,175.75 467.68,176.22 468.64,176.67C469.59,177.12 470.56,177.55 471.53,177.97C472.5,178.38 473.48,178.78 474.47,179.16C475.46,179.53 476.46,179.89 477.46,180.22C478.46,180.55 479.47,180.86 480.49,181.15C481.51,181.44 482.53,181.7 483.56,181.94C484.59,182.19 485.63,182.41 486.67,182.6C487.71,182.79 488.75,182.94 489.8,183.07C490.85,183.2 491.9,183.3 492.96,183.37C494.01,183.44 495.07,183.48 496.13,183.48C497.18,183.48 498.24,183.44 499.3,183.37C500.35,183.29 501.4,183.17 502.45,183.02C503.49,182.86 504.54,182.67 505.57,182.45C506.6,182.22 507.63,181.96 508.64,181.66C509.65,181.36 510.65,181.02 511.64,180.64C512.63,180.26 513.6,179.84 514.56,179.4C515.51,178.95 516.46,178.47 517.38,177.95C518.3,177.44 519.21,176.89 520.08,176.3C520.96,175.71 521.82,175.09 522.65,174.43C523.48,173.78 524.28,173.09 525.06,172.38C525.84,171.66 526.59,170.91 527.3,170.14C528.02,169.36 528.71,168.56 529.37,167.73C530.02,166.9 530.65,166.05 531.24,165.17C531.83,164.3 532.38,163.39 532.9,162.47C533.42,161.55 533.9,160.61 534.35,159.65C534.8,158.7 535.22,157.73 535.6,156.74C535.98,155.75 536.33,154.75 536.63,153.74C536.94,152.73 537.2,151.7 537.43,150.67C537.66,149.64 537.85,148.6 538.01,147.55C538.16,146.51 538.29,145.46 538.37,144.4C538.45,143.35 538.49,142.29 538.5,141.24C538.51,140.18 538.48,139.12 538.42,138.07C538.36,137.01 538.27,135.96 538.16,134.91C538.04,133.86 537.9,132.81 537.73,131.76C537.56,130.72 537.35,129.68 537.13,128.65C536.91,127.62 536.66,126.59 536.4,125.56C536.14,124.54 535.86,123.52 535.56,122.5C535.26,121.49 534.94,120.48 534.61,119.48C534.27,118.48 533.92,117.48 533.55,116.49C533.18,115.5 532.8,114.51 532.4,113.53C532.01,112.55 531.6,111.57 531.18,110.6C530.76,109.63 530.33,108.67 529.89,107.71C529.45,106.75 529,105.79 528.54,104.84C528.07,103.89 527.6,102.94 527.11,102C526.63,101.07 526.13,100.13 525.63,99.2C525.12,98.27 524.61,97.35 524.08,96.43C523.55,95.52 523.02,94.6 522.48,93.7C521.93,92.79 521.38,91.89 520.82,90.99C520.26,90.09 519.69,89.2 519.11,88.32C518.53,87.44 517.94,86.56 517.34,85.69C516.74,84.82 516.13,83.96 515.5,83.1C514.88,82.25 514.25,81.4 513.59,80.57C512.94,79.74 512.28,78.91 511.59,78.11C510.91,77.3 510.22,76.5 509.5,75.72C508.79,74.95 508.05,74.18 507.3,73.44C506.55,72.69 505.78,71.97 504.99,71.26C504.21,70.56 503.4,69.87 502.57,69.21C501.75,68.55 500.91,67.92 500.04,67.31C499.18,66.7 498.3,66.11 497.4,65.55C496.5,65 495.58,64.48 494.64,64C493.7,63.51 492.75,63.04 491.78,62.62C490.81,62.2 489.82,61.82 488.82,61.48C487.82,61.13 486.81,60.83 485.79,60.56C484.77,60.29 483.73,60.05 482.7,59.86C481.66,59.67 480.61,59.51 479.56,59.41C478.51,59.3 477.45,59.25 476.39,59.24C475.34,59.23 474.28,59.27 473.22,59.34C472.17,59.42 471.11,59.53 470.07,59.69C469.03,59.85 467.99,60.06 466.96,60.31C465.94,60.56 464.92,60.86 463.92,61.19C462.91,61.52 461.92,61.89 460.94,62.3C459.97,62.7 459.01,63.14 458.06,63.62C457.12,64.1 456.19,64.61 455.29,65.16C454.39,65.71 453.5,66.29 452.64,66.9C451.78,67.52 450.94,68.16 450.13,68.84C449.32,69.51 448.52,70.21 447.76,70.94C446.99,71.67 446.25,72.42 445.53,73.2C444.81,73.98 444.12,74.78 443.45,75.6C442.78,76.41 442.14,77.26 441.52,78.11C440.91,78.97 440.31,79.85 439.74,80.74C439.17,81.63 438.62,82.53 438.1,83.45C437.57,84.37 437.08,85.3 436.6,86.24C436.12,87.18 435.65,88.14 435.22,89.1C434.78,90.06 434.37,91.04 433.98,92.02C433.59,93 433.22,93.99 432.87,94.99C432.53,95.99 432.22,97 431.92,98.02C431.63,99.03 431.36,100.06 431.12,101.08C430.87,102.11 430.65,103.15 430.45,104.18Z"></path><path fill="none" stroke="#161311" stroke-width="4" d="M165.85,298.65C165.55,299.68 165.3,300.73 165.09,301.77C164.89,302.82 164.72,303.88 164.6,304.94C164.47,306.01 164.38,307.08 164.35,308.14C164.31,309.21 164.33,310.28 164.38,311.35C164.43,312.42 164.52,313.49 164.65,314.55C164.77,315.61 164.94,316.67 165.14,317.72C165.34,318.77 165.58,319.82 165.86,320.85C166.13,321.88 166.45,322.91 166.8,323.92C167.14,324.93 167.51,325.94 167.91,326.93C168.32,327.92 168.75,328.9 169.22,329.86C169.68,330.83 170.17,331.78 170.69,332.71C171.21,333.65 171.77,334.56 172.36,335.46C172.94,336.35 173.55,337.24 174.19,338.09C174.82,338.95 175.49,339.79 176.18,340.61C176.86,341.43 177.58,342.23 178.31,343.01C179.05,343.78 179.81,344.54 180.6,345.27C181.38,345.99 182.19,346.7 183.02,347.38C183.84,348.05 184.7,348.7 185.57,349.32C186.44,349.94 187.33,350.54 188.24,351.1C189.15,351.67 190.07,352.21 191.01,352.72C191.96,353.23 192.91,353.71 193.88,354.16C194.85,354.61 195.84,355.03 196.83,355.42C197.83,355.81 198.84,356.18 199.86,356.5C200.87,356.83 201.9,357.13 202.94,357.38C203.98,357.64 205.03,357.86 206.08,358.04C207.14,358.23 208.2,358.38 209.26,358.49C210.32,358.6 211.39,358.69 212.46,358.71C213.53,358.74 214.6,358.72 215.67,358.66C216.74,358.6 217.81,358.5 218.87,358.36C219.93,358.22 220.99,358.04 222.03,357.81C223.07,357.58 224.11,357.3 225.12,356.96C226.14,356.63 227.14,356.25 228.12,355.82C229.1,355.4 230.07,354.93 231,354.41C231.94,353.89 232.85,353.33 233.73,352.72C234.61,352.11 235.46,351.46 236.28,350.77C237.09,350.08 237.88,349.35 238.62,348.58C239.37,347.82 240.08,347.01 240.75,346.18C241.42,345.35 242.05,344.48 242.64,343.59C243.23,342.7 243.78,341.78 244.28,340.83C244.78,339.89 245.23,338.91 245.63,337.92C246.03,336.93 246.4,335.92 246.7,334.9C247,333.87 247.25,332.82 247.46,331.78C247.66,330.73 247.83,329.67 247.95,328.61C248.08,327.54 248.17,326.47 248.2,325.41C248.24,324.34 248.22,323.27 248.17,322.2C248.12,321.13 248.03,320.06 247.9,319C247.78,317.94 247.61,316.88 247.41,315.83C247.21,314.78 246.97,313.73 246.69,312.7C246.42,311.67 246.1,310.64 245.75,309.63C245.41,308.62 245.04,307.61 244.64,306.62C244.23,305.63 243.8,304.65 243.33,303.69C242.87,302.72 242.38,301.77 241.86,300.84C241.34,299.9 240.78,298.99 240.19,298.09C239.61,297.2 239,296.31 238.36,295.46C237.73,294.6 237.06,293.76 236.37,292.94C235.69,292.12 234.97,291.32 234.24,290.54C233.5,289.77 232.74,289.01 231.95,288.28C231.17,287.56 230.36,286.85 229.53,286.17C228.71,285.5 227.85,284.85 226.98,284.23C226.11,283.61 225.22,283.01 224.31,282.45C223.4,281.88 222.48,281.34 221.54,280.83C220.59,280.32 219.64,279.84 218.67,279.39C217.7,278.94 216.71,278.52 215.72,278.13C214.72,277.74 213.71,277.37 212.69,277.05C211.68,276.72 210.65,276.42 209.61,276.17C208.57,275.91 207.52,275.69 206.47,275.51C205.41,275.32 204.35,275.17 203.29,275.06C202.23,274.95 201.16,274.86 200.09,274.84C199.02,274.81 197.95,274.83 196.88,274.89C195.81,274.95 194.74,275.05 193.68,275.19C192.62,275.33 191.56,275.51 190.52,275.74C189.48,275.97 188.44,276.25 187.43,276.59C186.41,276.92 185.41,277.3 184.43,277.73C183.45,278.15 182.48,278.62 181.55,279.14C180.61,279.66 179.7,280.22 178.82,280.83C177.94,281.44 177.09,282.09 176.27,282.78C175.46,283.47 174.67,284.2 173.93,284.97C173.18,285.73 172.47,286.54 171.8,287.37C171.13,288.2 170.5,289.07 169.91,289.96C169.32,290.85 168.77,291.77 168.27,292.72C167.77,293.66 167.32,294.64 166.92,295.63C166.52,296.62 166.15,297.63 165.85,298.65Z"></path><path fill="none" stroke="#161311" stroke-width="4" d="M501.5,314.15C501.39,315.22 501.3,316.3 501.22,317.38C501.14,318.46 501.07,319.54 501.01,320.62C500.96,321.7 500.92,322.78 500.88,323.86C500.85,324.94 500.83,326.02 500.81,327.1C500.8,328.18 500.8,329.26 500.8,330.34C500.81,331.42 500.83,332.5 500.87,333.58C500.9,334.67 500.94,335.75 500.99,336.83C501.04,337.91 501.1,338.99 501.17,340.06C501.25,341.14 501.34,342.22 501.44,343.3C501.54,344.37 501.66,345.45 501.78,346.52C501.91,347.59 502.04,348.67 502.2,349.74C502.35,350.81 502.52,351.88 502.71,352.94C502.9,354.01 503.09,355.07 503.31,356.13C503.53,357.19 503.76,358.24 504.01,359.29C504.26,360.34 504.53,361.39 504.83,362.43C505.12,363.47 505.44,364.51 505.77,365.53C506.11,366.56 506.46,367.58 506.84,368.6C507.21,369.61 507.6,370.62 508.02,371.62C508.44,372.61 508.87,373.6 509.33,374.58C509.79,375.56 510.28,376.53 510.78,377.49C511.28,378.44 511.81,379.39 512.35,380.32C512.9,381.25 513.46,382.18 514.05,383.08C514.64,383.99 515.25,384.88 515.89,385.75C516.53,386.63 517.19,387.48 517.87,388.32C518.55,389.16 519.26,389.98 519.99,390.78C520.71,391.58 521.46,392.36 522.23,393.12C523,393.88 523.79,394.62 524.6,395.33C525.41,396.04 526.25,396.73 527.1,397.4C527.95,398.07 528.81,398.71 529.7,399.33C530.59,399.95 531.49,400.54 532.41,401.11C533.33,401.67 534.28,402.21 535.23,402.72C536.18,403.24 537.14,403.73 538.12,404.19C539.09,404.65 540.09,405.08 541.09,405.49C542.09,405.89 543.11,406.27 544.13,406.61C545.15,406.96 546.19,407.28 547.23,407.57C548.27,407.87 549.32,408.13 550.37,408.36C551.43,408.59 552.5,408.78 553.56,408.94C554.63,409.1 555.71,409.23 556.78,409.33C557.86,409.42 558.94,409.49 560.02,409.52C561.1,409.55 562.18,409.55 563.26,409.51C564.34,409.48 565.42,409.4 566.5,409.3C567.57,409.2 568.65,409.08 569.72,408.91C570.78,408.74 571.85,408.54 572.9,408.31C573.96,408.08 575.01,407.8 576.05,407.51C577.09,407.22 578.12,406.89 579.14,406.54C580.16,406.19 581.18,405.81 582.18,405.4C583.18,404.99 584.17,404.55 585.14,404.09C586.12,403.62 587.08,403.13 588.03,402.61C588.98,402.09 589.91,401.55 590.83,400.98C591.75,400.42 592.66,399.82 593.54,399.2C594.43,398.58 595.29,397.93 596.14,397.26C596.98,396.59 597.81,395.89 598.62,395.18C599.43,394.46 600.22,393.72 600.98,392.96C601.75,392.19 602.5,391.41 603.22,390.61C603.94,389.8 604.64,388.98 605.32,388.14C606,387.3 606.66,386.44 607.29,385.56C607.93,384.69 608.54,383.8 609.13,382.89C609.71,381.98 610.27,381.05 610.81,380.12C611.36,379.19 611.88,378.24 612.38,377.28C612.88,376.32 613.36,375.35 613.82,374.37C614.27,373.39 614.7,372.4 615.11,371.4C615.52,370.4 615.91,369.39 616.28,368.37C616.65,367.36 616.99,366.33 617.32,365.3C617.64,364.27 617.95,363.23 618.22,362.19C618.5,361.14 618.74,360.09 618.97,359.03C619.2,357.98 619.4,356.91 619.59,355.85C619.77,354.78 619.94,353.72 620.08,352.64C620.23,351.57 620.35,350.5 620.45,349.42C620.55,348.35 620.63,347.27 620.69,346.19C620.75,345.11 620.8,344.03 620.82,342.95C620.85,341.87 620.86,340.78 620.84,339.7C620.83,338.62 620.8,337.54 620.75,336.46C620.71,335.38 620.65,334.3 620.57,333.22C620.49,332.15 620.39,331.07 620.28,329.99C620.16,328.92 620.03,327.85 619.88,326.77C619.74,325.7 619.57,324.63 619.39,323.57C619.21,322.5 619.02,321.44 618.81,320.38C618.59,319.32 618.35,318.27 618.1,317.21C617.85,316.16 617.58,315.11 617.3,314.07C617.03,313.03 616.73,311.98 616.42,310.95C616.11,309.91 615.79,308.88 615.45,307.85C615.12,306.83 614.77,305.8 614.41,304.78C614.05,303.76 613.68,302.75 613.3,301.74C612.91,300.73 612.52,299.72 612.12,298.71C611.72,297.71 611.3,296.71 610.87,295.72C610.45,294.73 610.01,293.74 609.56,292.76C609.11,291.77 608.65,290.79 608.18,289.82C607.7,288.85 607.22,287.88 606.73,286.92C606.24,285.96 605.73,285 605.21,284.05C604.69,283.11 604.16,282.16 603.62,281.23C603.07,280.29 602.52,279.37 601.94,278.45C601.37,277.54 600.78,276.63 600.17,275.73C599.57,274.84 598.94,273.95 598.3,273.08C597.67,272.21 597.01,271.35 596.34,270.51C595.66,269.66 594.97,268.83 594.26,268.01C593.55,267.2 592.82,266.4 592.07,265.62C591.32,264.84 590.55,264.08 589.77,263.34C588.98,262.6 588.17,261.88 587.35,261.18C586.52,260.48 585.68,259.8 584.81,259.15C583.95,258.51 583.06,257.88 582.16,257.29C581.26,256.69 580.34,256.13 579.4,255.59C578.46,255.06 577.5,254.55 576.53,254.08C575.56,253.61 574.56,253.18 573.56,252.78C572.56,252.38 571.54,252.01 570.51,251.69C569.47,251.37 568.43,251.09 567.37,250.86C566.32,250.62 565.25,250.43 564.18,250.28C563.11,250.12 562.04,250.01 560.96,249.93C559.88,249.85 558.8,249.81 557.72,249.81C556.64,249.81 555.56,249.86 554.48,249.95C553.4,250.03 552.33,250.15 551.26,250.3C550.19,250.46 549.12,250.64 548.06,250.87C547.01,251.1 545.96,251.37 544.92,251.67C543.88,251.97 542.85,252.3 541.84,252.66C540.82,253.02 539.81,253.42 538.81,253.84C537.82,254.26 536.83,254.71 535.87,255.19C534.9,255.67 533.94,256.18 533.01,256.72C532.07,257.26 531.16,257.84 530.26,258.44C529.36,259.03 528.47,259.66 527.61,260.31C526.75,260.96 525.9,261.64 525.08,262.34C524.26,263.04 523.45,263.76 522.67,264.51C521.89,265.26 521.13,266.03 520.4,266.82C519.66,267.61 518.94,268.42 518.25,269.25C517.56,270.08 516.89,270.93 516.24,271.8C515.6,272.67 514.98,273.56 514.39,274.46C513.79,275.36 513.22,276.28 512.66,277.2C512.11,278.13 511.57,279.07 511.06,280.03C510.55,280.98 510.07,281.95 509.6,282.92C509.13,283.89 508.68,284.88 508.26,285.87C507.83,286.86 507.43,287.87 507.05,288.88C506.66,289.89 506.3,290.91 505.96,291.94C505.62,292.96 505.3,294 505,295.03C504.7,296.07 504.42,297.12 504.15,298.16C503.89,299.21 503.66,300.27 503.43,301.33C503.21,302.38 503.01,303.45 502.82,304.51C502.63,305.57 502.45,306.64 502.29,307.71C502.13,308.78 501.98,309.85 501.85,310.92C501.72,312 501.61,313.07 501.5,314.15Z"></path></g><g class="ink-knockout"></g></g></svg> </div></div><div class="post-content-shell"><article class="content"><div class="inkling-intro-cta tinker-cta-row"><a class="btn primary" href="https://tinker.thinkingmachines.ai/playground?utm_source=blog&amp;utm_campaign=inkling_small_model_release" target="_blank" rel="noopener" data-astro-cid-ekguhzzh="true">Try on Tinker</a><a class="btn plain btn-with-chevron" href="https://thinkingmachines.ai/model-card/inkling-small/" data-astro-cid-ekguhzzh="true">Model card<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg" data-astro-cid-ekguhzzh=""><path fill-rule="evenodd" clip-rule="evenodd" d="M6.56701 3.34347C6.78189 3.14201 7.1194 3.1529 7.32086 3.36779L11.3208 7.63446C11.5131 7.8396 11.5131 8.15884 11.3208 8.36399L7.32086 12.6306C7.1194 12.8456 6.78189 12.8565 6.56701 12.655C6.35212 12.4535 6.34123 12.116 6.54269 11.9011L10.2007 7.99922L6.54269 4.09733C6.34123 3.88244 6.35212 3.54492 6.56701 3.34347Z" fill="currentColor" data-astro-cid-ekguhzzh=""></path></svg></a><a class="btn plain btn-with-chevron" href="https://huggingface.co/thinkingmachines/Inkling-Small" target="_blank" rel="noopener" data-astro-cid-ekguhzzh="true">Hugging Face<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg" data-astro-cid-ekguhzzh=""><path fill-rule="evenodd" clip-rule="evenodd" d="M6.56701 3.34347C6.78189 3.14201 7.1194 3.1529 7.32086 3.36779L11.3208 7.63446C11.5131 7.8396 11.5131 8.15884 11.3208 8.36399L7.32086 12.6306C7.1194 12.8456 6.78189 12.8565 6.56701 12.655C6.35212 12.4535 6.34123 12.116 6.54269 11.9011L10.2007 7.99922L6.54269 4.09733C6.34123 3.88244 6.35212 3.54492 6.56701 3.34347Z" fill="currentColor" data-astro-cid-ekguhzzh=""></path></svg></a></div> <p>Today, we are releasing Inkling-Small, an efficient open-weights model that achieves comparable performance to <a href="https://thinkingmachines.ai/news/introducing-inkling/">Inkling</a> at a quarter of its size.</p> <p>Inkling-Small is a Mixture-of-Experts transformer with 276B total parameters, 12B active, trained on NVIDIA GB300 NVL72 systems. Like Inkling, it features native reasoning over audio and images, variable thinking effort, a context window of up to 1M tokens, and well-rounded performance across a range of benchmarks.</p> <figure class="model-parameter-comparison"><div class="model-parameter-comparison-panels"><section class="model-parameter-panel model-card--inklingsmall" style="--waffle-accent: #8F3F71"><h3 class="model-parameter-title">Inkling-Small</h3><div class="model-card-waffle"><div class="model-card-waffle-grid" style="--waffle-cols: 25" role="img" data-waffle-sequence="0" aria-label="Inkling-Small: 276 billion total parameters, drawn as 57 of 200 blocks, of which 2 represent the 12 billion activated per token"><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span><span class="model-card-waffle-cell is-empty"></span></div></div><div class="model-parameter-specs" role="group" aria-label="Inkling-Small parameter values"><div class="model-card-spec-value model-card-spec-value--keyed"><span class="model-card-key"><span class="model-card-key-chip model-card-key-chip--total"></span>276B total</span> <span class="model-card-key"><span class="model-card-key-chip model-card-key-chip--active"></span>12B active</span></div></div></section><section class="model-parameter-panel model-card--inkling" style="--waffle-accent: #0155BF"><h3 class="model-parameter-title">Inkling</h3><div class="model-card-waffle"><div class="model-card-waffle-grid" style="--waffle-cols: 25" role="img" data-waffle-sequence="1" aria-label="Inkling: 975 billion total parameters, drawn as 200 of 200 blocks, of which 8 represent the 41 billion activated per token"><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-active"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span><span class="model-card-waffle-cell is-filled"></span></div></div><div class="model-parameter-specs" role="group" aria-label="Inkling parameter values"><div class="model-card-spec-value model-card-spec-value--keyed"><span class="model-card-key"><span class="model-card-key-chip model-card-key-chip--total"></span>975B total</span> <span class="model-card-key"><span class="model-card-key-chip model-card-key-chip--active"></span>41B active</span></div></div></section></div><figcaption>Parameter comparison between Inkling-Small and Inkling.</figcaption></figure> <p>Compared to Inkling, Inkling-Small achieves comparable performance with much less compute. Across agentic tool use (Terminal-Bench 2.1), reasoning (HLE text-only, no tools), and instruction following (IFBench) benchmarks, Inkling-Small is more efficient than Inkling and competitive with other models in its weight class. Furthermore, its <a href="https://thinkingmachines.ai/news/introducing-inkling/#controllable-thinking-effort">variable thinking effort</a> lets users easily adapt it to target their use case, balancing cost and performance.</p> <link rel="stylesheet" href="small-efficiency.css"> <figure class="small-efficiency-switcher" data-se-switch="true"><div class="se-toggle-row"><div class="demo-tabs" role="tablist" aria-label="Efficiency chart axis"><button class="demo-tab" type="button" role="tab" id="se-tab-flops" aria-controls="se-panel-flops" aria-selected="true" data-se-tab="flops">TFLOPs</button><button class="demo-tab" type="button" role="tab" id="se-tab-cost" aria-controls="se-panel-cost" aria-selected="false" tabindex="-1" data-se-tab="cost">Cost</button></div></div><div class="se-panels"><div class="se-panel" id="se-panel-flops" role="tabpanel" aria-labelledby="se-tab-flops" data-se-panel="flops"> <div class="ce-figure ce-figure--3 intelligence-interactivity-figure" data-plot-point-stagger="14" data-plot-point-jitter="8"> <div class="ce-legend intelligence-interactivity-legend" aria-label="Model legend"> <span class="ii-legend-item"><span class="ce-swatch ce-swatch--hero" aria-hidden="true" style="--ce-swatch-color: var(--ce-inklingsmall);"></span>Inkling-Small <span class="ce-legend-meta">12B active</span> <span class="ce-legend-meta">(effort sweep)</span></span> <span class="ii-legend-item"><span class="ce-swatch" aria-hidden="true" style="--ce-swatch-color: var(--ce-inkling);"></span>Inkling <span class="ce-legend-meta">41B active</span> <span class="ce-legend-meta">(effort sweep)</span></span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--ce-peer);" aria-hidden="true"></span>Comparison models</span> </div> <div class="benchmark-results-controls chart-hscroll-controls" role="group" aria-label="performance-compute chart controls"> <button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll performance-compute chart left" title="Scroll left"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg> </button> <button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll performance-compute chart right" title="Scroll right"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg> </button> </div> <div class="ce-scroll intelligence-interactivity-charts chart-hscroll"> <div class="ce-chart intelligence-interactivity-chart" role="img" aria-label="Three panels: pass@1 on Terminal-Bench 2.1, Humanity's Last Exam (Text-Only) and IFBench vs approximate output TFLOPs per task, log axis."> <svg class="ce-svg intelligence-interactivity-svg" viewBox="0 24 1332 420" aria-hidden="true" focusable="false"> <g class="ce-panel"> <text class="ce-title ii-panel-title" x="246" y="46">Terminal-Bench 2.1</text> <g class="ii-grid"> <line x1="96" y1="372.0" x2="396" y2="372.0"></line> <line x1="96" y1="272.7" x2="396" y2="272.7"></line> <line x1="96" y1="173.3" x2="396" y2="173.3"></line> <line x1="96" y1="74.0" x2="396" y2="74.0"></line> <line x1="108.6" y1="74" x2="108.6" y2="372"></line> <line x1="147.9" y1="74" x2="147.9" y2="372"></line> <line x1="187.1" y1="74" x2="187.1" y2="372"></line> <line x1="239.0" y1="74" x2="239.0" y2="372"></line> <line x1="278.3" y1="74" x2="278.3" y2="372"></line> <line x1="317.5" y1="74" x2="317.5" y2="372"></line> <line x1="369.4" y1="74" x2="369.4" y2="372"></line> </g> <g class="ii-axis"><line x1="96" y1="74" x2="96" y2="372"></line><line x1="96" y1="372" x2="396" y2="372"></line></g> <g class="ii-ticks ii-y-ticks"> <text x="86" y="376.5">20%</text> <text x="86" y="277.2">40%</text> <text x="86" y="177.8">60%</text> <text x="86" y="78.5">80%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="108.6" y="396">50</text> <text x="147.9" y="396">100</text> <text x="187.1" y="396">200</text> <text x="239.0" y="396">500</text> <text x="278.3" y="396">1k</text> <text x="317.5" y="396">2k</text> <text x="369.4" y="396">5k</text> </g> <polyline class="ce-line ce-inkling ce-base-series" style="--ce-series-delay:55ms;" points="219.7,293.5 245.8,248.2 276.9,190.9 303.6,174.8 337.2,167.2 356.2,169.3"></polyline> <polyline class="ce-line ce-inklingsmall ce-hero-line ce-hero-series" style="--ce-series-delay:0ms;" points="120.2,317.2 112.5,331.1 157.2,270.4 217.3,193.7 287.1,147.6"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(235.9 279.7)" data-ii-tooltip-title="Nemotron 3 Super" tabindex="0" role="img" aria-label="Nemotron 3 Super"><title>Nemotron 3 Super</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(350.8 190.2)" data-ii-tooltip-title="Nemotron 3 Ultra" tabindex="0" role="img" aria-label="Nemotron 3 Ultra"><title>Nemotron 3 Ultra</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(260.2 164.4)" data-ii-tooltip-title="DeepSeek V4 Flash" tabindex="0" role="img" aria-label="DeepSeek V4 Flash"><title>DeepSeek V4 Flash</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(232.6 216.5)" data-ii-tooltip-title="Qwen3.5-397B-A17B" tabindex="0" role="img" aria-label="Qwen3.5-397B-A17B"><title>Qwen3.5-397B-A17B</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(226.9 155.1)" data-ii-tooltip-title="MiMo V2.5" tabindex="0" role="img" aria-label="MiMo V2.5"><title>MiMo V2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(205.5 196.0)" data-ii-tooltip-title="Minimax M2.7" tabindex="0" role="img" aria-label="Minimax M2.7"><title>Minimax M2.7</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(288.4 225.5)" data-ii-tooltip-title="Kimi K2.5" tabindex="0" role="img" aria-label="Kimi K2.5"><title>Kimi K2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(334.6 122.7)" data-ii-tooltip-title="Kimi K2.6" tabindex="0" role="img" aria-label="Kimi K2.6"><title>Kimi K2.6</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(384.8 84.5)" data-ii-tooltip-title="GLM 5.2" tabindex="0" role="img" aria-label="GLM 5.2"><title>GLM 5.2</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(219.7 293.5)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(245.8 248.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(276.9 190.9)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(303.6 174.8)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(337.2 167.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(356.2 169.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(120.2 317.2)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(112.5 331.1)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(157.2 270.4)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(217.3 193.7)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(287.1 147.6)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> </g> <g class="ce-labels"> <text class="ce-label ce-label--inklingsmall" x="120.2" y="305.2" text-anchor="middle">minimal</text> <text class="ce-label ce-label--inklingsmall" x="112.5" y="347.1" text-anchor="middle">low</text> <text class="ce-label ce-label--inklingsmall" x="157.2" y="258.4" text-anchor="middle">medium</text> <text class="ce-label ce-label--inklingsmall" x="217.3" y="181.7" text-anchor="middle">high</text> <text class="ce-label ce-label--inklingsmall" x="287.1" y="135.6" text-anchor="middle">xhigh</text> </g> </g> <g class="ce-panel"> <text class="ce-title ii-panel-title" x="666" y="46">Humanity's Last Exam (Text-Only)</text> <g class="ii-grid"> <line x1="516" y1="372.0" x2="816" y2="372.0"></line> <line x1="516" y1="301.0" x2="816" y2="301.0"></line> <line x1="516" y1="230.1" x2="816" y2="230.1"></line> <line x1="516" y1="159.1" x2="816" y2="159.1"></line> <line x1="516" y1="88.2" x2="816" y2="88.2"></line> <line x1="541.4" y1="74" x2="541.4" y2="372"></line> <line x1="587.0" y1="74" x2="587.0" y2="372"></line> <line x1="621.4" y1="74" x2="621.4" y2="372"></line> <line x1="655.9" y1="74" x2="655.9" y2="372"></line> <line x1="701.5" y1="74" x2="701.5" y2="372"></line> <line x1="736.0" y1="74" x2="736.0" y2="372"></line> <line x1="770.4" y1="74" x2="770.4" y2="372"></line> <line x1="816.0" y1="74" x2="816.0" y2="372"></line> </g> <g class="ii-axis"><line x1="516" y1="74" x2="516" y2="372"></line><line x1="516" y1="372" x2="816" y2="372"></line></g> <g class="ii-ticks ii-y-ticks"> <text x="506" y="376.5">0%</text> <text x="506" y="305.5">10%</text> <text x="506" y="234.6">20%</text> <text x="506" y="163.6">30%</text> <text x="506" y="92.7">40%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="541.4" y="396">20</text> <text x="587.0" y="396">50</text> <text x="621.4" y="396">100</text> <text x="655.9" y="396">200</text> <text x="701.5" y="396">500</text> <text x="736.0" y="396">1k</text> <text x="770.4" y="396">2k</text> <text x="816.0" y="396">5k</text> </g> <polyline class="ce-line ce-inkling ce-base-series" style="--ce-series-delay:55ms;" points="645.1,292.6 666.6,274.1 686.2,254.3 728.5,213.1 752.9,194.0 773.1,161.3"></polyline> <polyline class="ce-line ce-inklingsmall ce-hero-line ce-hero-series" style="--ce-series-delay:0ms;" points="528.7,318.4 534.6,313.2 611.2,252.0 689.0,177.9 727.4,147.8"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(726.5 235.9)" data-ii-tooltip-title="Nemotron 3 Super" tabindex="0" role="img" aria-label="Nemotron 3 Super"><title>Nemotron 3 Super</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(805.8 180.4)" data-ii-tooltip-title="Nemotron 3 Ultra" tabindex="0" role="img" aria-label="Nemotron 3 Ultra"><title>Nemotron 3 Ultra</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(766.6 144.5)" data-ii-tooltip-title="DeepSeek V4 Flash" tabindex="0" role="img" aria-label="DeepSeek V4 Flash"><title>DeepSeek V4 Flash</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(722.3 178.4)" data-ii-tooltip-title="Qwen3.5-397B-A17B" tabindex="0" role="img" aria-label="Qwen3.5-397B-A17B"><title>Qwen3.5-397B-A17B</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(711.4 193.5)" data-ii-tooltip-title="MiMo V2.5" tabindex="0" role="img" aria-label="MiMo V2.5"><title>MiMo V2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(701.7 172.7)" data-ii-tooltip-title="Minimax M2.7" tabindex="0" role="img" aria-label="Minimax M2.7"><title>Minimax M2.7</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(760.2 163.4)" data-ii-tooltip-title="Kimi K2.5" tabindex="0" role="img" aria-label="Kimi K2.5"><title>Kimi K2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(786.3 117.3)" data-ii-tooltip-title="Kimi K2.6" tabindex="0" role="img" aria-label="Kimi K2.6"><title>Kimi K2.6</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(795.0 88.2)" data-ii-tooltip-title="GLM 5.2" tabindex="0" role="img" aria-label="GLM 5.2"><title>GLM 5.2</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(663.8 240.7)" data-ii-tooltip-title="gpt-oss-120b" tabindex="0" role="img" aria-label="gpt-oss-120b"><title>gpt-oss-120b</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(645.1 292.6)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(666.6 274.1)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(686.2 254.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(728.5 213.1)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(752.9 194.0)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(773.1 161.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(528.7 318.4)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(534.6 313.2)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(611.2 252.0)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(689.0 177.9)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(727.4 147.8)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> </g> <g class="ce-labels"> <text class="ce-label ce-label--inklingsmall" x="528.7" y="306.4" text-anchor="middle">minimal</text> <text class="ce-label ce-label--inklingsmall" x="534.6" y="329.2" text-anchor="middle">low</text> <text class="ce-label ce-label--inklingsmall" x="611.2" y="240.0" text-anchor="middle">medium</text> <text class="ce-label ce-label--inklingsmall" x="689.0" y="165.9" text-anchor="middle">high</text> <text class="ce-label ce-label--inklingsmall" x="727.4" y="135.8" text-anchor="middle">xhigh</text> </g> </g> <g class="ce-panel"> <text class="ce-title ii-panel-title" x="1086" y="46">IFBench</text> <g class="ii-grid"> <line x1="936" y1="372.0" x2="1236" y2="372.0"></line> <line x1="936" y1="272.7" x2="1236" y2="272.7"></line> <line x1="936" y1="173.3" x2="1236" y2="173.3"></line> <line x1="936" y1="74.0" x2="1236" y2="74.0"></line> <line x1="958.5" y1="74" x2="958.5" y2="372"></line> <line x1="1030.0" y1="74" x2="1030.0" y2="372"></line> <line x1="1084.1" y1="74" x2="1084.1" y2="372"></line> <line x1="1138.2" y1="74" x2="1138.2" y2="372"></line> <line x1="1209.7" y1="74" x2="1209.7" y2="372"></line> </g> <g class="ii-axis"><line x1="936" y1="74" x2="936" y2="372"></line><line x1="936" y1="372" x2="1236" y2="372"></line></g> <g class="ii-ticks ii-y-ticks"> <text x="926" y="376.5">60%</text> <text x="926" y="277.2">70%</text> <text x="926" y="177.8">80%</text> <text x="926" y="78.5">90%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="958.5" y="396">20</text> <text x="1030.0" y="396">50</text> <text x="1084.1" y="396">100</text> <text x="1138.2" y="396">200</text> <text x="1209.7" y="396">500</text> </g> <polyline class="ce-line ce-inkling ce-base-series" style="--ce-series-delay:55ms;" points="1047.5,272.7 1085.2,224.0 1121.6,193.2 1163.3,180.2 1190.6,169.3 1204.8,181.3 1212.8,174.3"></polyline> <polyline class="ce-line ce-inklingsmall ce-hero-line ce-hero-series" style="--ce-series-delay:0ms;" points="956.2,209.5 970.9,189.6 1046.2,153.4 1079.8,140.6 1105.1,139.8"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(1096.7 257.8)" data-ii-tooltip-title="Nemotron 3 Super" tabindex="0" role="img" aria-label="Nemotron 3 Super"><title>Nemotron 3 Super</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1216.7 159.8)" data-ii-tooltip-title="Nemotron 3 Ultra" tabindex="0" role="img" aria-label="Nemotron 3 Ultra"><title>Nemotron 3 Ultra</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1183.6 181.4)" data-ii-tooltip-title="DeepSeek V4 Flash" tabindex="0" role="img" aria-label="DeepSeek V4 Flash"><title>DeepSeek V4 Flash</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1173.2 185.5)" data-ii-tooltip-title="Qwen3.5-397B-A17B" tabindex="0" role="img" aria-label="Qwen3.5-397B-A17B"><title>Qwen3.5-397B-A17B</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1194.4 301.1)" data-ii-tooltip-title="MiMo V2.5" tabindex="0" role="img" aria-label="MiMo V2.5"><title>MiMo V2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1167.4 215.9)" data-ii-tooltip-title="Minimax M2.7" tabindex="0" role="img" aria-label="Minimax M2.7"><title>Minimax M2.7</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1188.6 272.7)" data-ii-tooltip-title="Kimi K2.5" tabindex="0" role="img" aria-label="Kimi K2.5"><title>Kimi K2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1216.3 213.1)" data-ii-tooltip-title="Kimi K2.6" tabindex="0" role="img" aria-label="Kimi K2.6"><title>Kimi K2.6</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1213.3 239.8)" data-ii-tooltip-title="GLM 5.2" tabindex="0" role="img" aria-label="GLM 5.2"><title>GLM 5.2</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1056.1 282.8)" data-ii-tooltip-title="gpt-oss-120b" tabindex="0" role="img" aria-label="gpt-oss-120b"><title>gpt-oss-120b</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1047.5 272.7)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1085.2 224.0)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1121.6 193.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1163.3 180.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1190.6 169.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1204.8 181.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1212.8 174.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(956.2 209.5)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(970.9 189.6)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(1046.2 153.4)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(1079.8 140.6)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(1105.1 139.8)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> </g> <g class="ce-labels"> <text class="ce-label ce-label--inklingsmall" x="956.2" y="197.5" text-anchor="middle">minimal</text> <text class="ce-label ce-label--inklingsmall" x="970.9" y="177.6" text-anchor="middle">low</text> <text class="ce-label ce-label--inklingsmall" x="1046.2" y="141.4" text-anchor="middle">medium</text> <text class="ce-label ce-label--inklingsmall" x="1087.8" y="128.6" text-anchor="start">high</text> <text class="ce-label ce-label--inklingsmall" x="1105.1" y="155.8" text-anchor="middle">xhigh</text> </g> </g> <text class="ce-ylabel ii-axis-label" x="40" y="223" transform="rotate(-90 40 223)">pass@1</text> <text class="ce-xlabel ii-axis-label" x="666" y="428">Approx. Output TFLOPs / task</text> </svg> </div> </div> </div> </div><div class="se-panel" id="se-panel-cost" role="tabpanel" aria-labelledby="se-tab-cost" data-se-panel="cost" hidden=""> <div class="ce-figure ce-figure--3 intelligence-interactivity-figure" data-plot-point-stagger="14" data-plot-point-jitter="8"> <div class="ce-legend intelligence-interactivity-legend" aria-label="Model legend"> <span class="ii-legend-item"><span class="ce-swatch ce-swatch--hero" aria-hidden="true" style="--ce-swatch-color: var(--ce-inklingsmall);"></span>Inkling-Small <span class="ce-legend-meta">12B active</span> <span class="ce-legend-meta">(effort sweep)</span></span> <span class="ii-legend-item"><span class="ce-swatch" aria-hidden="true" style="--ce-swatch-color: var(--ce-inkling);"></span>Inkling <span class="ce-legend-meta">41B active</span> <span class="ce-legend-meta">(effort sweep)</span></span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--ce-peer);" aria-hidden="true"></span>Comparison models</span> </div> <div class="benchmark-results-controls chart-hscroll-controls" role="group" aria-label="performance-cost chart controls"> <button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll performance-cost chart left" title="Scroll left"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg> </button> <button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll performance-cost chart right" title="Scroll right"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg> </button> </div> <div class="ce-scroll intelligence-interactivity-charts chart-hscroll"> <div class="ce-chart intelligence-interactivity-chart" role="img" aria-label="Three panels: pass@1 on Terminal-Bench 2.1, Humanity's Last Exam (Text-Only) and IFBench vs estimated output cost per sample, log axis."> <svg class="ce-svg intelligence-interactivity-svg" viewBox="0 24 1332 420" aria-hidden="true" focusable="false"> <g class="ce-panel"> <text class="ce-title ii-panel-title" x="246" y="46">Terminal-Bench 2.1</text> <g class="ii-grid"> <line x1="96" y1="372.0" x2="396" y2="372.0"></line> <line x1="96" y1="272.7" x2="396" y2="272.7"></line> <line x1="96" y1="173.3" x2="396" y2="173.3"></line> <line x1="96" y1="74.0" x2="396" y2="74.0"></line> <line x1="96.0" y1="74" x2="96.0" y2="372"></line> <line x1="143.9" y1="74" x2="143.9" y2="372"></line> <line x1="180.1" y1="74" x2="180.1" y2="372"></line> <line x1="216.4" y1="74" x2="216.4" y2="372"></line> <line x1="264.2" y1="74" x2="264.2" y2="372"></line> <line x1="300.5" y1="74" x2="300.5" y2="372"></line> <line x1="336.7" y1="74" x2="336.7" y2="372"></line> <line x1="384.6" y1="74" x2="384.6" y2="372"></line> </g> <g class="ii-axis"><line x1="96" y1="74" x2="96" y2="372"></line><line x1="96" y1="372" x2="396" y2="372"></line></g> <g class="ii-ticks ii-y-ticks"> <text x="86" y="376.5">20%</text> <text x="86" y="277.2">40%</text> <text x="86" y="177.8">60%</text> <text x="86" y="78.5">80%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="96.0" y="396">0.2c</text> <text x="143.9" y="396">0.5c</text> <text x="180.1" y="396">1c</text> <text x="216.4" y="396">2c</text> <text x="264.2" y="396">5c</text> <text x="300.5" y="396">10c</text> <text x="336.7" y="396">20c</text> <text x="384.6" y="396">50c</text> </g> <polyline class="ce-line ce-inkling ce-base-series" style="--ce-series-delay:55ms;" points="209.7,293.5 233.6,248.1 262.4,190.9 287.1,174.9 318.1,167.2 335.5,169.3"></polyline> <polyline class="ce-line ce-inklingsmall ce-hero-line ce-hero-series" style="--ce-series-delay:0ms;" points="118.4,317.2 111.2,331.1 152.6,270.4 208.0,193.7 272.4,147.7"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(201.0 279.7)" data-ii-tooltip-title="Nemotron 3 Super" tabindex="0" role="img" aria-label="Nemotron 3 Super"><title>Nemotron 3 Super</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(283.3 190.2)" data-ii-tooltip-title="Nemotron 3 Ultra" tabindex="0" role="img" aria-label="Nemotron 3 Ultra"><title>Nemotron 3 Ultra</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(167.3 164.4)" data-ii-tooltip-title="DeepSeek V4 Flash" tabindex="0" role="img" aria-label="DeepSeek V4 Flash"><title>DeepSeek V4 Flash</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(243.9 216.5)" data-ii-tooltip-title="Qwen3.5-397B-A17B" tabindex="0" role="img" aria-label="Qwen3.5-397B-A17B"><title>Qwen3.5-397B-A17B</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(129.1 155.1)" data-ii-tooltip-title="MiMo V2.5" tabindex="0" role="img" aria-label="MiMo V2.5"><title>MiMo V2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(206.7 196.0)" data-ii-tooltip-title="Minimax M2.7" tabindex="0" role="img" aria-label="Minimax M2.7"><title>Minimax M2.7</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(270.3 225.5)" data-ii-tooltip-title="Kimi K2.5" tabindex="0" role="img" aria-label="Kimi K2.5"><title>Kimi K2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(327.9 122.7)" data-ii-tooltip-title="Kimi K2.6" tabindex="0" role="img" aria-label="Kimi K2.6"><title>Kimi K2.6</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(367.5 84.4)" data-ii-tooltip-title="GLM 5.2" tabindex="0" role="img" aria-label="GLM 5.2"><title>GLM 5.2</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(209.7 293.5)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(233.6 248.1)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(262.4 190.9)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(287.1 174.9)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(318.1 167.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(335.5 169.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(118.4 317.2)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(111.2 331.1)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(152.6 270.4)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(208.0 193.7)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(272.4 147.7)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> </g> <g class="ce-labels"> <text class="ce-label ce-label--inklingsmall" x="118.4" y="305.2" text-anchor="middle">minimal</text> <text class="ce-label ce-label--inklingsmall" x="111.2" y="347.1" text-anchor="middle">low</text> <text class="ce-label ce-label--inklingsmall" x="152.6" y="258.4" text-anchor="middle">medium</text> <text class="ce-label ce-label--inklingsmall" x="208.0" y="181.7" text-anchor="middle">high</text> <text class="ce-label ce-label--inklingsmall" x="272.4" y="135.7" text-anchor="middle">xhigh</text> </g> </g> <g class="ce-panel"> <text class="ce-title ii-panel-title" x="666" y="46">Humanity's Last Exam (Text-Only)</text> <g class="ii-grid"> <line x1="516" y1="372.0" x2="816" y2="372.0"></line> <line x1="516" y1="301.0" x2="816" y2="301.0"></line> <line x1="516" y1="230.1" x2="816" y2="230.1"></line> <line x1="516" y1="159.1" x2="816" y2="159.1"></line> <line x1="516" y1="88.2" x2="816" y2="88.2"></line> <line x1="544.1" y1="74" x2="544.1" y2="372"></line> <line x1="577.3" y1="74" x2="577.3" y2="372"></line> <line x1="621.1" y1="74" x2="621.1" y2="372"></line> <line x1="654.2" y1="74" x2="654.2" y2="372"></line> <line x1="687.4" y1="74" x2="687.4" y2="372"></line> <line x1="731.2" y1="74" x2="731.2" y2="372"></line> <line x1="764.3" y1="74" x2="764.3" y2="372"></line> <line x1="797.5" y1="74" x2="797.5" y2="372"></line> </g> <g class="ii-axis"><line x1="516" y1="74" x2="516" y2="372"></line><line x1="516" y1="372" x2="816" y2="372"></line></g> <g class="ii-ticks ii-y-ticks"> <text x="506" y="376.5">0%</text> <text x="506" y="305.5">10%</text> <text x="506" y="234.6">20%</text> <text x="506" y="163.6">30%</text> <text x="506" y="92.7">40%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="544.1" y="396">0.1c</text> <text x="577.3" y="396">0.2c</text> <text x="621.1" y="396">0.5c</text> <text x="654.2" y="396">1c</text> <text x="687.4" y="396">2c</text> <text x="731.2" y="396">5c</text> <text x="764.3" y="396">10c</text> <text x="797.5" y="396">20c</text> </g> <polyline class="ce-line ce-inkling ce-base-series" style="--ce-series-delay:55ms;" points="643.4,292.5 663.9,274.1 682.8,254.2 723.4,213.1 746.9,193.9 766.3,161.3"></polyline> <polyline class="ce-line ce-inklingsmall ce-hero-line ce-hero-series" style="--ce-series-delay:0ms;" points="534.8,318.4 536.5,313.3 611.5,251.9 686.1,177.9 723.0,147.8"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(700.0 235.9)" data-ii-tooltip-title="Nemotron 3 Super" tabindex="0" role="img" aria-label="Nemotron 3 Super"><title>Nemotron 3 Super</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(754.5 180.4)" data-ii-tooltip-title="Nemotron 3 Ultra" tabindex="0" role="img" aria-label="Nemotron 3 Ultra"><title>Nemotron 3 Ultra</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(687.3 144.5)" data-ii-tooltip-title="DeepSeek V4 Flash" tabindex="0" role="img" aria-label="DeepSeek V4 Flash"><title>DeepSeek V4 Flash</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(737.9 178.3)" data-ii-tooltip-title="Qwen3.5-397B-A17B" tabindex="0" role="img" aria-label="Qwen3.5-397B-A17B"><title>Qwen3.5-397B-A17B</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(627.4 193.5)" data-ii-tooltip-title="MiMo V2.5" tabindex="0" role="img" aria-label="MiMo V2.5"><title>MiMo V2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(707.0 172.8)" data-ii-tooltip-title="Minimax M2.7" tabindex="0" role="img" aria-label="Minimax M2.7"><title>Minimax M2.7</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(751.4 163.4)" data-ii-tooltip-title="Kimi K2.5" tabindex="0" role="img" aria-label="Kimi K2.5"><title>Kimi K2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(790.3 117.3)" data-ii-tooltip-title="Kimi K2.6" tabindex="0" role="img" aria-label="Kimi K2.6"><title>Kimi K2.6</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(792.6 88.2)" data-ii-tooltip-title="GLM 5.2" tabindex="0" role="img" aria-label="GLM 5.2"><title>GLM 5.2</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(669.6 240.7)" data-ii-tooltip-title="gpt-oss-120b" tabindex="0" role="img" aria-label="gpt-oss-120b"><title>gpt-oss-120b</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(643.4 292.5)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(663.9 274.1)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(682.8 254.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(723.4 213.1)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(746.9 193.9)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(766.3 161.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(534.8 318.4)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(536.5 313.3)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(611.5 251.9)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(686.1 177.9)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(723.0 147.8)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> </g> <g class="ce-labels"> <text class="ce-label ce-label--inklingsmall" x="534.8" y="306.4" text-anchor="middle">minimal</text> <text class="ce-label ce-label--inklingsmall" x="536.5" y="329.3" text-anchor="middle">low</text> <text class="ce-label ce-label--inklingsmall" x="611.5" y="239.9" text-anchor="middle">medium</text> <text class="ce-label ce-label--inklingsmall" x="686.1" y="165.9" text-anchor="middle">high</text> <text class="ce-label ce-label--inklingsmall" x="723.0" y="135.8" text-anchor="middle">xhigh</text> </g> </g> <g class="ce-panel"> <text class="ce-title ii-panel-title" x="1086" y="46">IFBench</text> <g class="ii-grid"> <line x1="936" y1="372.0" x2="1236" y2="372.0"></line> <line x1="936" y1="272.7" x2="1236" y2="272.7"></line> <line x1="936" y1="173.3" x2="1236" y2="173.3"></line> <line x1="936" y1="74.0" x2="1236" y2="74.0"></line> <line x1="954.5" y1="74" x2="954.5" y2="372"></line> <line x1="998.4" y1="74" x2="998.4" y2="372"></line> <line x1="1056.5" y1="74" x2="1056.5" y2="372"></line> <line x1="1100.4" y1="74" x2="1100.4" y2="372"></line> <line x1="1144.3" y1="74" x2="1144.3" y2="372"></line> <line x1="1202.4" y1="74" x2="1202.4" y2="372"></line> </g> <g class="ii-axis"><line x1="936" y1="74" x2="936" y2="372"></line><line x1="936" y1="372" x2="1236" y2="372"></line></g> <g class="ii-ticks ii-y-ticks"> <text x="926" y="376.5">60%</text> <text x="926" y="277.2">70%</text> <text x="926" y="177.8">80%</text> <text x="926" y="78.5">90%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="954.5" y="396">0.1c</text> <text x="998.4" y="396">0.2c</text> <text x="1056.5" y="396">0.5c</text> <text x="1100.4" y="396">1c</text> <text x="1144.3" y="396">2c</text> <text x="1202.4" y="396">5c</text> </g> <polyline class="ce-line ce-inkling ce-base-series" style="--ce-series-delay:55ms;" points="1026.1,272.7 1056.6,224.0 1086.2,193.2 1120.0,180.3 1142.2,169.4 1153.7,181.3 1160.2,174.3"></polyline> <polyline class="ce-line ce-inklingsmall ce-hero-line ce-hero-series" style="--ce-series-delay:0ms;" points="952.4,209.5 964.5,189.6 1025.7,153.3 1053.0,140.6 1073.6,139.8"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(1037.5 257.8)" data-ii-tooltip-title="Nemotron 3 Super" tabindex="0" role="img" aria-label="Nemotron 3 Super"><title>Nemotron 3 Super</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1106.0 159.8)" data-ii-tooltip-title="Nemotron 3 Ultra" tabindex="0" role="img" aria-label="Nemotron 3 Ultra"><title>Nemotron 3 Ultra</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1040.0 181.4)" data-ii-tooltip-title="DeepSeek V4 Flash" tabindex="0" role="img" aria-label="DeepSeek V4 Flash"><title>DeepSeek V4 Flash</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1155.3 185.5)" data-ii-tooltip-title="Qwen3.5-397B-A17B" tabindex="0" role="img" aria-label="Qwen3.5-397B-A17B"><title>Qwen3.5-397B-A17B</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1039.7 301.0)" data-ii-tooltip-title="MiMo V2.5" tabindex="0" role="img" aria-label="MiMo V2.5"><title>MiMo V2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1135.7 215.9)" data-ii-tooltip-title="Minimax M2.7" tabindex="0" role="img" aria-label="Minimax M2.7"><title>Minimax M2.7</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1137.2 272.7)" data-ii-tooltip-title="Kimi K2.5" tabindex="0" role="img" aria-label="Kimi K2.5"><title>Kimi K2.5</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1178.0 213.1)" data-ii-tooltip-title="Kimi K2.6" tabindex="0" role="img" aria-label="Kimi K2.6"><title>Kimi K2.6</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1167.4 239.9)" data-ii-tooltip-title="GLM 5.2" tabindex="0" role="img" aria-label="GLM 5.2"><title>GLM 5.2</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1044.1 282.8)" data-ii-tooltip-title="gpt-oss-120b" tabindex="0" role="img" aria-label="gpt-oss-120b"><title>gpt-oss-120b</title><circle class="ii-point-marker ce-peer" r="3.8"></circle></g> <g class="ii-point" transform="translate(1026.1 272.7)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1056.6 224.0)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1086.2 193.2)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1120.0 180.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1142.2 169.4)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1153.7 181.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(1160.2 174.3)"><circle class="ii-point-marker ce-inkling" r="3.8"></circle></g> <g class="ii-point" transform="translate(952.4 209.5)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(964.5 189.6)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(1025.7 153.3)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(1053.0 140.6)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> <g class="ii-point" transform="translate(1073.6 139.8)"><circle class="ii-point-marker ce-inklingsmall" r="4.2"></circle></g> </g> <g class="ce-labels"> <text class="ce-label ce-label--inklingsmall" x="952.4" y="197.5" text-anchor="middle">minimal</text> <text class="ce-label ce-label--inklingsmall" x="964.5" y="177.6" text-anchor="middle">low</text> <text class="ce-label ce-label--inklingsmall" x="1025.7" y="141.3" text-anchor="middle">medium</text> <text class="ce-label ce-label--inklingsmall" x="1053.0" y="128.6" text-anchor="middle">high</text> <text class="ce-label ce-label--inklingsmall" x="1081.6" y="127.8" text-anchor="start">xhigh</text> </g> </g> <text class="ce-ylabel ii-axis-label" x="40" y="223" transform="rotate(-90 40 223)">pass@1</text> <text class="ce-xlabel ii-axis-label" x="666" y="428">Estimated output cost ($) / sample</text> </svg> </div> </div> </div> </div></div><figcaption class="small-efficiency-figure-caption"><span class="se-cap" data-se-cap="flops" data-figure-number="2a">Performance-Compute Comparison. Sweeping <a href="https://thinkingmachines.ai/news/introducing-inkling/#controllable-thinking-effort">reasoning effort</a> from <a href="https://tinker-docs.thinkingmachines.ai/cookbook/inkling/thinking-effort/">minimal to xhigh</a> traces the performance-compute curve (output TFLOPs per sample) for Inkling-Small and Inkling on Terminal-Bench 2.1, HLE (no tool), and IFBench. We show that Inkling-Small is competitive with other open-weights models in a similar size range on both performance and efficiency. Output TFLOPs per sample are estimated as 2 × active parameters × mean generated tokens per sample, where generated-token counts include reasoning tokens and come from our evaluations or public reports from <a href="https://artificialanalysis.ai/evaluations/">Artificial Analysis</a>.</span><span class="se-cap" data-se-cap="cost" data-figure-number="2b" hidden="">Performance-Cost Comparison. Sweeping <a href="https://thinkingmachines.ai/news/introducing-inkling/#controllable-thinking-effort">reasoning effort</a> from <a href="https://tinker-docs.thinkingmachines.ai/cookbook/inkling/thinking-effort/">minimal to xhigh</a> traces the performance-cost curve (dollar output price per sample) for Inkling-Small and Inkling on Terminal-Bench 2.1, HLE (no tool), and IFBench. We show that Inkling-Small is competitive with other open-weights models in a similar size range on both performance and efficiency. Estimated output cost per sample is computed as mean generated tokens per sample × output price per token, where generated-token counts include reasoning tokens and come from our evaluations or public reports from <a href="https://artificialanalysis.ai/evaluations/">Artificial Analysis</a> (reasoning + answer tokens only). Inkling output pricing is $4.05 / 1M tokens and Inkling-Small output pricing is $1.20 / 1M tokens; for comparison-model pricing, we use the model provider’s official pricing when possible. When this is not possible, we use pricing from the recommended third party inference provider or pricing from Artificial Analysis, whichever is lower.</span></figcaption></figure> <p>We are releasing the <a href="https://huggingface.co/thinkingmachines/Inkling-Small">full weights</a> of Inkling-Small. We’re also making it available for fine-tuning on Tinker, and for text, image, and audio chat on <a href="https://tinker.thinkingmachines.ai/playground?utm_source=blog&amp;utm_campaign=inkling_small_model_release">Tinker Playground</a>.</p> <h2 id="capabilities">Capabilities</h2> <p>As we build our model family, we are always iterating on our approach. Inkling-Small began training after its larger counterpart, which let us improve its training process. For example, we made changes to Inkling-Small’s pre-training data mix and machine learning recipe. Additionally, we post-trained an earlier checkpoint, <a href="https://thinkingmachines.ai/news/introducing-inkling/#inkling-small">Inkling-Small (preview)</a>, in part using on-policy distillation with Inkling as the teacher. Starting from that checkpoint, we continued scaling agentic coding RL for two weeks. With these improvements, Inkling-Small surpassed Inkling on reasoning and agentic coding benchmarks. Inkling maintains an advantage on knowledge coverage and factuality.</p> <figure><div class="benchmark-radar-figure"><div class="benchmark-radar" id="benchmark-radar-small"><div class="benchmark-radar-scroll"><svg class="benchmark-radar-svg" data-benchmark-radar="" viewBox="0 0 860 600" role="img" aria-label="Inkling benchmark comparison" aria-describedby="benchmark-radar-desc-small"><desc id="benchmark-radar-desc-small">Spider chart comparing Inkling-Small, Inkling, DeepSeek V4 Flash, Gemini 3.5 Flash-Lite, and GPT 5.6 Luna on ten evaluations scored from zero to one hundred. Inkling-Small is shown with a solid purple line, Inkling with the heavier cobalt line, and the comparison models with dashed lines. Evaluations without a reported model score are plotted at zero. Hover an evaluation to compare every model's score.</desc><g data-radar-grid=""></g><g data-radar-axes=""></g><g data-radar-series=""></g><g data-radar-labels=""></g></svg></div><div class="benchmark-radar-legend" aria-label="Model legend"><span class="benchmark-radar-legend-pill" data-radar-legend-pill="" aria-hidden="true"></span><button class="benchmark-radar-legend-item" type="button" data-radar-model="inklingsmall"><span class="benchmark-radar-key benchmark-radar-key--inklingsmall benchmark-radar-key--hero" aria-hidden="true"></span>Inkling-Small</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="inkling"><span class="benchmark-radar-key benchmark-radar-key--inkling" aria-hidden="true"></span>Inkling</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="deepseek"><span class="benchmark-radar-key benchmark-radar-key--deepseek" aria-hidden="true"></span>DeepSeek V4 Flash</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="gemini"><span class="benchmark-radar-key benchmark-radar-key--gemini" aria-hidden="true"></span>Gemini 3.5 Flash-Lite</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="luna"><span class="benchmark-radar-key benchmark-radar-key--luna" aria-hidden="true"></span>GPT 5.6 Luna</button></div><div class="benchmark-radar-tooltip tooltip" data-radar-tooltip="" role="tooltip" aria-hidden="true"><div class="benchmark-radar-tooltip-title" data-radar-tooltip-title=""></div><div data-radar-tooltip-rows=""></div></div></div></div><figcaption>Inkling-Small is a broad, balanced generalist model. Benchmark scores are shown on a shared 0–100 scale; higher is better.</figcaption></figure> <h3 id="reasoning-and-agentic-tasks">Reasoning and Agentic Tasks</h3> <p>Inkling-Small matches or exceeds Inkling on reasoning and agentic tasks. On Humanity’s Last Exam it scores 31.6%, ahead of Inkling’s 29.7%, and the advantage holds at every thinking budget: Inkling-Small’s test-time compute curves sit above Inkling’s throughout. On SWEBench-Verified it exceeds 80%.</p> <p>Across many reasoning and agentic benchmarks, Inkling-Small at max reasoning effort has a strong performance-token tradeoff when compared to open weights models in its weight class.</p> <figure> <div class="cq-figure intelligence-interactivity-figure" data-plot-point-stagger="14" data-plot-point-jitter="12"> <div class="cq-legend intelligence-interactivity-legend" aria-label="Model legend"> <span class="ii-legend-item"><span class="cq-legend-dot" style="--cq-color: var(--cq-inklingsmall);" aria-hidden="true"></span>Inkling-Small</span> <span class="ii-legend-item"><span class="cq-legend-dot" style="--cq-color: var(--cq-inkling);" aria-hidden="true"></span>Inkling</span> <span class="ii-legend-item"><span class="cq-legend-dot" style="--cq-color: var(--cq-peer);" aria-hidden="true"></span>Comparison models</span> <span class="ii-legend-item"><span class="cq-frontier-key" aria-hidden="true"></span>Pareto frontier</span> </div> <div class="cq-grid-panels intelligence-interactivity-charts"> <div class="cq-cell intelligence-interactivity-chart"> <svg class="cq-svg intelligence-interactivity-svg" viewBox="0 0 460 300" role="img" aria-label="GDPval-AA v2 Elo against output tokens per task. Inkling-Small and Inkling are highlighted; the dashed line marks the non-dominated Pareto frontier."> <text class="cq-title" x="252" y="22" text-anchor="middle">GDPval-AA v2 (Elo)</text> <g class="cq-grid"> <line x1="56" x2="448" y1="272" y2="272"></line> <line x1="56" x2="448" y1="201.9" y2="201.9"></line> <line x1="56" x2="448" y1="131.9" y2="131.9"></line> <line x1="56" x2="448" y1="61.8" y2="61.8"></line> <line x1="69.9" x2="69.9" y1="44" y2="272"></line> <line x1="219.2" x2="219.2" y1="44" y2="272"></line> <line x1="368.4" x2="368.4" y1="44" y2="272"></line> </g> <g class="cq-axis"> <line x1="56" x2="56" y1="44" y2="272"></line> <line x1="56" x2="448" y1="272" y2="272"></line> </g> <g class="cq-ticks"> <text class="cq-tick cq-tick--y" x="48" y="276">900</text> <text class="cq-tick cq-tick--y" x="48" y="205.9">1100</text> <text class="cq-tick cq-tick--y" x="48" y="135.9">1300</text> <text class="cq-tick cq-tick--y" x="48" y="65.8">1500</text> <text class="cq-tick" x="69.9" y="292">15k</text> <text class="cq-tick" x="219.2" y="292">30k</text> <text class="cq-tick" x="368.4" y="292">60k</text> </g> <polyline class="cq-frontier" points="77,250.3 82.7,233.8 83.5,186.2 160.5,144.1 162.1,142.7 231.4,129.8 297.1,100.1 427,56.9"></polyline> <g class="cq-points"> <circle class="cq-point ii-point cq-point--inklingsmall" cx="162.1" cy="142.7" r="5.6" data-ii-tooltip-title="Inkling-Small" data-ii-tooltip-values="GDPval-AA v2 · 1269 · 23k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Inkling-Small, 1269, 23k output tokens per task"><title>GDPval-AA v2 · Inkling-Small: 1269, 23k output tokens/task</title></circle> <circle class="cq-point ii-point cq-point--inkling" cx="208.9" cy="153.6" r="5.6" data-ii-tooltip-title="Inkling" data-ii-tooltip-values="GDPval-AA v2 · 1238 · 28.6k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Inkling, 1238, 28.6k output tokens per task"><title>GDPval-AA v2 · Inkling: 1238, 28.6k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="198.4" cy="179.5" r="4.2" data-ii-tooltip-title="Nemotron 3 Ultra" data-ii-tooltip-values="GDPval-AA v2 · 1164 · 27.2k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Nemotron 3 Ultra, 1164, 27.2k output tokens per task"><title>GDPval-AA v2 · Nemotron 3 Ultra: 1164, 27.2k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="205.9" cy="170.8" r="4.2" data-ii-tooltip-title="DeepSeek V4 Flash" data-ii-tooltip-values="GDPval-AA v2 · 1189 · 28.2k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: DeepSeek V4 Flash, 1189, 28.2k output tokens per task"><title>GDPval-AA v2 · DeepSeek V4 Flash: 1189, 28.2k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="231.4" cy="129.8" r="4.2" data-ii-tooltip-title="DeepSeek V4 Pro" data-ii-tooltip-values="GDPval-AA v2 · 1306 · 31.8k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: DeepSeek V4 Pro, 1306, 31.8k output tokens per task"><title>GDPval-AA v2 · DeepSeek V4 Pro: 1306, 31.8k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="77" cy="250.3" r="4.2" data-ii-tooltip-title="Qwen3.5-397B-A17B" data-ii-tooltip-values="GDPval-AA v2 · 962 · 15.5k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Qwen3.5-397B-A17B, 962, 15.5k output tokens per task"><title>GDPval-AA v2 · Qwen3.5-397B-A17B: 962, 15.5k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="83.5" cy="186.2" r="4.2" data-ii-tooltip-title="MiMo V2.5" data-ii-tooltip-values="GDPval-AA v2 · 1145 · 16k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: MiMo V2.5, 1145, 16k output tokens per task"><title>GDPval-AA v2 · MiMo V2.5: 1145, 16k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="160.5" cy="144.1" r="4.2" data-ii-tooltip-title="MiMo V2.5 Pro" data-ii-tooltip-values="GDPval-AA v2 · 1265.1 · 22.8k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: MiMo V2.5 Pro, 1265.1, 22.8k output tokens per task"><title>GDPval-AA v2 · MiMo V2.5 Pro: 1265.1, 22.8k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="171.6" cy="181.3" r="4.2" data-ii-tooltip-title="Minimax M2.7" data-ii-tooltip-values="GDPval-AA v2 · 1159 · 24.1k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Minimax M2.7, 1159, 24.1k output tokens per task"><title>GDPval-AA v2 · Minimax M2.7: 1159, 24.1k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="297.1" cy="100.1" r="4.2" data-ii-tooltip-title="MiniMax M3" data-ii-tooltip-values="GDPval-AA v2 · 1390.8 · 43.1k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: MiniMax M3, 1390.8, 43.1k output tokens per task"><title>GDPval-AA v2 · MiniMax M3: 1390.8, 43.1k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="82.7" cy="233.8" r="4.2" data-ii-tooltip-title="Kimi K2.5" data-ii-tooltip-values="GDPval-AA v2 · 1009 · 15.9k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Kimi K2.5, 1009, 15.9k output tokens per task"><title>GDPval-AA v2 · Kimi K2.5: 1009, 15.9k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="182.4" cy="170.4" r="4.2" data-ii-tooltip-title="Kimi K2.6" data-ii-tooltip-values="GDPval-AA v2 · 1190 · 25.3k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: Kimi K2.6, 1190, 25.3k output tokens per task"><title>GDPval-AA v2 · Kimi K2.6: 1190, 25.3k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="427" cy="56.9" r="4.2" data-ii-tooltip-title="GLM 5.2" data-ii-tooltip-values="GDPval-AA v2 · 1514 · 78.8k tokens/task" tabindex="0" role="img" aria-label="GDPval-AA v2: GLM 5.2, 1514, 78.8k output tokens per task"><title>GDPval-AA v2 · GLM 5.2: 1514, 78.8k output tokens/task</title></circle> </g> <g class="cq-labels"> <text class="cq-label cq-label--inklingsmall" x="162.1" y="130.1" text-anchor="middle">Inkling-Small</text> <text class="cq-label cq-label--inkling" x="220.5" y="157.6" text-anchor="start">Inkling</text> </g> </svg> <p class="cq-cell-xlabel">Output tokens/task</p> </div> <div class="cq-cell intelligence-interactivity-chart"> <svg class="cq-svg intelligence-interactivity-svg" viewBox="0 0 460 300" role="img" aria-label="Tau-cubed Banking score against output tokens per task. Inkling-Small and Inkling are highlighted; the dashed line marks the non-dominated Pareto frontier."> <text class="cq-title" x="252" y="22" text-anchor="middle">τ³-Banking</text> <g class="cq-grid"> <line x1="56" x2="448" y1="272" y2="272"></line> <line x1="56" x2="448" y1="226.4" y2="226.4"></line> <line x1="56" x2="448" y1="180.8" y2="180.8"></line> <line x1="56" x2="448" y1="135.2" y2="135.2"></line> <line x1="56" x2="448" y1="89.6" y2="89.6"></line> <line x1="56" x2="448" y1="44" y2="44"></line> <line x1="75.2" x2="75.2" y1="44" y2="272"></line> <line x1="254.6" x2="254.6" y1="44" y2="272"></line> <line x1="434" x2="434" y1="44" y2="272"></line> </g> <g class="cq-axis"> <line x1="56" x2="56" y1="44" y2="272"></line> <line x1="56" x2="448" y1="272" y2="272"></line> </g> <g class="cq-ticks"> <text class="cq-tick cq-tick--y" x="48" y="276">5%</text> <text class="cq-tick cq-tick--y" x="48" y="230.4">10%</text> <text class="cq-tick cq-tick--y" x="48" y="184.8">15%</text> <text class="cq-tick cq-tick--y" x="48" y="139.2">20%</text> <text class="cq-tick cq-tick--y" x="48" y="93.6">25%</text> <text class="cq-tick cq-tick--y" x="48" y="48">30%</text> <text class="cq-tick" x="75.2" y="292">5k</text> <text class="cq-tick" x="254.6" y="292">10k</text> <text class="cq-tick" x="434" y="292">20k</text> </g> <polyline class="cq-frontier" points="77,199 79.1,176.2 210.5,101.5 251.8,80.5 427,73.2"></polyline> <g class="cq-points"> <circle class="cq-point ii-point cq-point--inklingsmall" cx="79.1" cy="176.2" r="5.6" data-ii-tooltip-title="Inkling-Small" data-ii-tooltip-values="τ³-Banking · 15.5% · 5.1k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Inkling-Small, 15.5%, 5.1k output tokens per task"><title>τ³-Banking · Inkling-Small: 15.5%, 5.1k output tokens/task</title></circle> <circle class="cq-point ii-point cq-point--inkling" cx="210.5" cy="101.5" r="5.6" data-ii-tooltip-title="Inkling" data-ii-tooltip-values="τ³-Banking · 23.7% · 8.4k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Inkling, 23.7%, 8.4k output tokens per task"><title>τ³-Banking · Inkling: 23.7%, 8.4k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="242.3" cy="191.7" r="4.2" data-ii-tooltip-title="Nemotron 3 Ultra" data-ii-tooltip-values="τ³-Banking · 13.8% · 9.5k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Nemotron 3 Ultra, 13.8%, 9.5k output tokens per task"><title>τ³-Banking · Nemotron 3 Ultra: 13.8%, 9.5k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="281.8" cy="108.8" r="4.2" data-ii-tooltip-title="DeepSeek V4 Flash" data-ii-tooltip-values="τ³-Banking · 22.9% · 11.1k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: DeepSeek V4 Flash, 22.9%, 11.1k output tokens per task"><title>τ³-Banking · DeepSeek V4 Flash: 22.9%, 11.1k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="251.8" cy="80.5" r="4.2" data-ii-tooltip-title="DeepSeek V4 Pro" data-ii-tooltip-values="τ³-Banking · 26.0% · 9.9k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: DeepSeek V4 Pro, 26.0%, 9.9k output tokens per task"><title>τ³-Banking · DeepSeek V4 Pro: 26.0%, 9.9k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="187.3" cy="195.4" r="4.2" data-ii-tooltip-title="Qwen3.5-397B-A17B" data-ii-tooltip-values="τ³-Banking · 13.4% · 7.7k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Qwen3.5-397B-A17B, 13.4%, 7.7k output tokens per task"><title>τ³-Banking · Qwen3.5-397B-A17B: 13.4%, 7.7k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="150.4" cy="257.4" r="4.2" data-ii-tooltip-title="MiMo V2.5" data-ii-tooltip-values="τ³-Banking · 6.6% · 6.7k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: MiMo V2.5, 6.6%, 6.7k output tokens per task"><title>τ³-Banking · MiMo V2.5: 6.6%, 6.7k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="168.6" cy="235.5" r="4.2" data-ii-tooltip-title="MiMo V2.5 Pro" data-ii-tooltip-values="τ³-Banking · 9.0% · 7.2k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: MiMo V2.5 Pro, 9.0%, 7.2k output tokens per task"><title>τ³-Banking · MiMo V2.5 Pro: 9.0%, 7.2k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="113.9" cy="236.4" r="4.2" data-ii-tooltip-title="Minimax M2.7" data-ii-tooltip-values="τ³-Banking · 8.9% · 5.8k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Minimax M2.7, 8.9%, 5.8k output tokens per task"><title>τ³-Banking · Minimax M2.7: 8.9%, 5.8k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="77" cy="199" r="4.2" data-ii-tooltip-title="MiniMax M3" data-ii-tooltip-values="τ³-Banking · 13.0% · 5k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: MiniMax M3, 13.0%, 5k output tokens per task"><title>τ³-Banking · MiniMax M3: 13.0%, 5k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="103.8" cy="188.1" r="4.2" data-ii-tooltip-title="Kimi K2.5" data-ii-tooltip-values="τ³-Banking · 14.2% · 5.6k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Kimi K2.5, 14.2%, 5.6k output tokens per task"><title>τ³-Banking · Kimi K2.5: 14.2%, 5.6k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="274.9" cy="129.7" r="4.2" data-ii-tooltip-title="Kimi K2.6" data-ii-tooltip-values="τ³-Banking · 20.6% · 10.8k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: Kimi K2.6, 20.6%, 10.8k output tokens per task"><title>τ³-Banking · Kimi K2.6: 20.6%, 10.8k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="427" cy="73.2" r="4.2" data-ii-tooltip-title="GLM 5.2" data-ii-tooltip-values="τ³-Banking · 26.8% · 19.5k tokens/task" tabindex="0" role="img" aria-label="τ³-Banking: GLM 5.2, 26.8%, 19.5k output tokens per task"><title>τ³-Banking · GLM 5.2: 26.8%, 19.5k output tokens/task</title></circle> </g> <g class="cq-labels"> <text class="cq-label cq-label--inklingsmall" x="89.7" y="165.6" text-anchor="start">Inkling-Small</text> <text class="cq-label cq-label--inkling" x="198.9" y="105.5" text-anchor="end">Inkling</text> </g> </svg> <p class="cq-cell-xlabel">Output tokens/task</p> </div> <div class="cq-cell intelligence-interactivity-chart"> <svg class="cq-svg intelligence-interactivity-svg" viewBox="0 0 460 300" role="img" aria-label="AA-Briefcase Elo against output tokens per task. Inkling-Small and Inkling are highlighted; the dashed line marks the non-dominated Pareto frontier."> <text class="cq-title" x="252" y="22" text-anchor="middle">AA-Briefcase (Elo)</text> <g class="cq-grid"> <line x1="56" x2="448" y1="272" y2="272"></line> <line x1="56" x2="448" y1="196" y2="196"></line> <line x1="56" x2="448" y1="120" y2="120"></line> <line x1="56" x2="448" y1="44" y2="44"></line> <line x1="85.2" x2="85.2" y1="44" y2="272"></line> <line x1="261.4" x2="261.4" y1="44" y2="272"></line> <line x1="391.2" x2="391.2" y1="44" y2="272"></line> </g> <g class="cq-axis"> <line x1="56" x2="56" y1="44" y2="272"></line> <line x1="56" x2="448" y1="272" y2="272"></line> </g> <g class="cq-ticks"> <text class="cq-tick cq-tick--y" x="48" y="276">700</text> <text class="cq-tick cq-tick--y" x="48" y="200">900</text> <text class="cq-tick cq-tick--y" x="48" y="124">1100</text> <text class="cq-tick cq-tick--y" x="48" y="48">1300</text> <text class="cq-tick" x="85.2" y="292">30k</text> <text class="cq-tick" x="261.4" y="292">60k</text> <text class="cq-tick" x="391.2" y="292">100k</text> </g> <polyline class="cq-frontier" points="77,189.5 340.5,117 427,56.9"></polyline> <g class="cq-points"> <circle class="cq-point ii-point cq-point--inklingsmall" cx="77" cy="189.5" r="5.6" data-ii-tooltip-title="Inkling-Small" data-ii-tooltip-values="AA-Briefcase · 917 · 29k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: Inkling-Small, 917, 29k output tokens per task"><title>AA-Briefcase · Inkling-Small: 917, 29k output tokens/task</title></circle> <circle class="cq-point ii-point cq-point--inkling" cx="226.1" cy="219.2" r="5.6" data-ii-tooltip-title="Inkling" data-ii-tooltip-values="AA-Briefcase · 839 · 52.2k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: Inkling, 839, 52.2k output tokens per task"><title>AA-Briefcase · Inkling: 839, 52.2k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="202.2" cy="207.4" r="4.2" data-ii-tooltip-title="Nemotron 3 Ultra" data-ii-tooltip-values="AA-Briefcase · 870 · 47.5k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: Nemotron 3 Ultra, 870, 47.5k output tokens per task"><title>AA-Briefcase · Nemotron 3 Ultra: 870, 47.5k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="142.8" cy="221.5" r="4.2" data-ii-tooltip-title="DeepSeek V4 Flash" data-ii-tooltip-values="AA-Briefcase · 833 · 37.6k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: DeepSeek V4 Flash, 833, 37.6k output tokens per task"><title>AA-Briefcase · DeepSeek V4 Flash: 833, 37.6k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="108.2" cy="204" r="4.2" data-ii-tooltip-title="MiMo V2.5 Pro" data-ii-tooltip-values="AA-Briefcase · 878.9 · 32.8k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: MiMo V2.5 Pro, 878.9, 32.8k output tokens per task"><title>AA-Briefcase · MiMo V2.5 Pro: 878.9, 32.8k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="340.5" cy="117" r="4.2" data-ii-tooltip-title="MiniMax M3" data-ii-tooltip-values="AA-Briefcase · 1107.8 · 81.9k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: MiniMax M3, 1107.8, 81.9k output tokens per task"><title>AA-Briefcase · MiniMax M3: 1107.8, 81.9k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="427" cy="56.9" r="4.2" data-ii-tooltip-title="GLM 5.2" data-ii-tooltip-values="AA-Briefcase · 1266 · 115k tokens/task" tabindex="0" role="img" aria-label="AA-Briefcase: GLM 5.2, 1266, 115k output tokens per task"><title>AA-Briefcase · GLM 5.2: 1266, 115k output tokens/task</title></circle> </g> <g class="cq-labels"> <text class="cq-label cq-label--inklingsmall" x="88.6" y="193.5" text-anchor="start">Inkling-Small</text> <text class="cq-label cq-label--inkling" x="237.7" y="223.2" text-anchor="start">Inkling</text> </g> </svg> <p class="cq-cell-xlabel">Output tokens/task</p> </div> <div class="cq-cell intelligence-interactivity-chart"> <svg class="cq-svg intelligence-interactivity-svg" viewBox="0 0 460 300" role="img" aria-label="CritPt score against output tokens per task. Inkling-Small and Inkling are highlighted; the dashed line marks the non-dominated Pareto frontier."> <text class="cq-title" x="252" y="22" text-anchor="middle">CritPt</text> <g class="cq-grid"> <line x1="56" x2="448" y1="272" y2="272"></line> <line x1="56" x2="448" y1="226.4" y2="226.4"></line> <line x1="56" x2="448" y1="180.8" y2="180.8"></line> <line x1="56" x2="448" y1="135.2" y2="135.2"></line> <line x1="56" x2="448" y1="89.6" y2="89.6"></line> <line x1="56" x2="448" y1="44" y2="44"></line> <line x1="80.4" x2="80.4" y1="44" y2="272"></line> <line x1="276.3" x2="276.3" y1="44" y2="272"></line> <line x1="424.4" x2="424.4" y1="44" y2="272"></line> </g> <g class="cq-axis"> <line x1="56" x2="56" y1="44" y2="272"></line> <line x1="56" x2="448" y1="272" y2="272"></line> </g> <g class="cq-ticks"> <text class="cq-tick cq-tick--y" x="48" y="276">0%</text> <text class="cq-tick cq-tick--y" x="48" y="230.4">5%</text> <text class="cq-tick cq-tick--y" x="48" y="184.8">10%</text> <text class="cq-tick cq-tick--y" x="48" y="139.2">15%</text> <text class="cq-tick cq-tick--y" x="48" y="93.6">20%</text> <text class="cq-tick cq-tick--y" x="48" y="48">25%</text> <text class="cq-tick" x="80.4" y="292">40k</text> <text class="cq-tick" x="276.3" y="292">100k</text> <text class="cq-tick" x="424.4" y="292">200k</text> </g> <polyline class="cq-frontier" points="77,266.5 78.1,243.7 86.6,238.3 161.5,235.5 226.8,222.8 277.7,196.3 285.8,153.4 289.6,81.4"></polyline> <g class="cq-points"> <circle class="cq-point ii-point cq-point--inklingsmall" cx="277.7" cy="196.3" r="5.6" data-ii-tooltip-title="Inkling-Small" data-ii-tooltip-values="CritPt · 8.3% · 101k tokens/task" tabindex="0" role="img" aria-label="CritPt: Inkling-Small, 8.3%, 101k output tokens per task"><title>CritPt · Inkling-Small: 8.3%, 101k output tokens/task</title></circle> <circle class="cq-point ii-point cq-point--inkling" cx="226.8" cy="222.8" r="5.6" data-ii-tooltip-title="Inkling" data-ii-tooltip-values="CritPt · 5.4% · 79.3k tokens/task" tabindex="0" role="img" aria-label="CritPt: Inkling, 5.4%, 79.3k output tokens per task"><title>CritPt · Inkling: 5.4%, 79.3k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="235.2" cy="243.7" r="4.2" data-ii-tooltip-title="Nemotron 3 Ultra" data-ii-tooltip-values="CritPt · 3.1% · 82.5k tokens/task" tabindex="0" role="img" aria-label="CritPt: Nemotron 3 Ultra, 3.1%, 82.5k output tokens per task"><title>CritPt · Nemotron 3 Ultra: 3.1%, 82.5k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="427" cy="207.2" r="4.2" data-ii-tooltip-title="DeepSeek V4 Flash" data-ii-tooltip-values="CritPt · 7.1% · 202k tokens/task" tabindex="0" role="img" aria-label="CritPt: DeepSeek V4 Flash, 7.1%, 202k output tokens per task"><title>CritPt · DeepSeek V4 Flash: 7.1%, 202k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="285.8" cy="153.4" r="4.2" data-ii-tooltip-title="DeepSeek V4 Pro" data-ii-tooltip-values="CritPt · 13.0% · 105k tokens/task" tabindex="0" role="img" aria-label="CritPt: DeepSeek V4 Pro, 13.0%, 105k output tokens per task"><title>CritPt · DeepSeek V4 Pro: 13.0%, 105k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="106.3" cy="256.5" r="4.2" data-ii-tooltip-title="Qwen3.5-397B-A17B" data-ii-tooltip-values="CritPt · 1.7% · 45.2k tokens/task" tabindex="0" role="img" aria-label="CritPt: Qwen3.5-397B-A17B, 1.7%, 45.2k output tokens per task"><title>CritPt · Qwen3.5-397B-A17B: 1.7%, 45.2k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="86.6" cy="238.3" r="4.2" data-ii-tooltip-title="MiMo V2.5" data-ii-tooltip-values="CritPt · 3.7% · 41.2k tokens/task" tabindex="0" role="img" aria-label="CritPt: MiMo V2.5, 3.7%, 41.2k output tokens per task"><title>CritPt · MiMo V2.5: 3.7%, 41.2k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="161.5" cy="235.5" r="4.2" data-ii-tooltip-title="MiMo V2.5 Pro" data-ii-tooltip-values="CritPt · 4.0% · 58.4k tokens/task" tabindex="0" role="img" aria-label="CritPt: MiMo V2.5 Pro, 4.0%, 58.4k output tokens per task"><title>CritPt · MiMo V2.5 Pro: 4.0%, 58.4k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="77" cy="266.5" r="4.2" data-ii-tooltip-title="Minimax M2.7" data-ii-tooltip-values="CritPt · 0.6% · 39.4k tokens/task" tabindex="0" role="img" aria-label="CritPt: Minimax M2.7, 0.6%, 39.4k output tokens per task"><title>CritPt · Minimax M2.7: 0.6%, 39.4k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="222.9" cy="238.1" r="4.2" data-ii-tooltip-title="MiniMax M3" data-ii-tooltip-values="CritPt · 3.7% · 77.9k tokens/task" tabindex="0" role="img" aria-label="CritPt: MiniMax M3, 3.7%, 77.9k output tokens per task"><title>CritPt · MiniMax M3: 3.7%, 77.9k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="78.1" cy="243.7" r="4.2" data-ii-tooltip-title="Kimi K2.5" data-ii-tooltip-values="CritPt · 3.1% · 39.6k tokens/task" tabindex="0" role="img" aria-label="CritPt: Kimi K2.5, 3.1%, 39.6k output tokens per task"><title>CritPt · Kimi K2.5: 3.1%, 39.6k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="405.2" cy="199" r="4.2" data-ii-tooltip-title="Kimi K2.6" data-ii-tooltip-values="CritPt · 8.0% · 183k tokens/task" tabindex="0" role="img" aria-label="CritPt: Kimi K2.6, 8.0%, 183k output tokens per task"><title>CritPt · Kimi K2.6: 8.0%, 183k output tokens/task</title></circle> <circle class="cq-point ii-point" cx="289.6" cy="81.4" r="4.2" data-ii-tooltip-title="GLM 5.2" data-ii-tooltip-values="CritPt · 20.9% · 106k tokens/task" tabindex="0" role="img" aria-label="CritPt: GLM 5.2, 20.9%, 106k output tokens per task"><title>CritPt · GLM 5.2: 20.9%, 106k output tokens/task</title></circle> </g> <g class="cq-labels"> <text class="cq-label cq-label--inklingsmall" x="288.3" y="185.7" text-anchor="start">Inkling-Small</text> <text class="cq-label cq-label--inkling" x="238.4" y="226.8" text-anchor="start">Inkling</text> </g> </svg> <p class="cq-cell-xlabel">Output tokens/task</p> </div> </div> <p class="cq-xlabel">Output tokens/task</p> </div> <figcaption>Token-Efficiency Performance Tradeoff on Reasoning + Agentic Benchmarks. We evaluate Inkling-Small and Inkling (with max effort) along with other open-weights models on agentic and reasoning tasks (GDPval-AA v2, τ³-Banking, AA-Briefcase and CritPt) and show performance and output token length (including reasoning + answer). Inkling-Small is among the most efficient open weights models, marked by the dashed line. Results were obtained from our evaluation or reference data from Artificial Analysis.</figcaption></figure> <p>Inkling-Small also runs smoothly across a variety of coding and agent harnesses, making it a cost-efficient choice for coding and tool-use workflows.</p> <h3 id="multimodality">Multimodality</h3> <p>Like Inkling, we crafted Inkling-Small for audio intelligence, making it a good candidate for real-world audio applications. We also improved its ability to use Python for visual tasks. The model can combine visual reasoning with operations such as cropping, zooming, and programmatic image inspection, improving usability on documents and charts where relevant information may be small or difficult to inspect directly.</p> <p>Inkling-Small uses the same natively multimodal encoder-free architecture as Inkling. Audio is represented as dMel spectrograms, while images are divided into 40×40-pixel patches and transformed using a four-layer hMLP. Both are transformed via a light-weight embedding layer and processed jointly with text tokens. Inkling-Small nearly matches Inkling across most multimodal evaluations at a lower cost. It retains strong performance on visual reasoning, chart and diagram understanding, mathematical visual question answering, speech understanding, and longer-form audio reasoning.</p> <div class="inkling-eval-table-block" style="--benchmark-model-count: 7"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Inkling-Small multimodal comparison table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Inkling-Small multimodal comparison table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Inkling-Small multimodal comparison table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="7"></colgroup><thead><tr class="benchmark-model-group-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model-group" colspan="5" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Open weights</span></span></th><th class="benchmark-model-group" colspan="2" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Closed weights</span></span></th></tr><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling-Small</span></span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">MiMo V2.5</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Nemotron-3</span><span class="benchmark-model-name-line">Nano-Omni</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3-Omni</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3.5</span><span class="benchmark-model-name-line">397B-A17B</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3.5</span><span class="benchmark-model-name-line">Omni-Plus</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Gemini 3.5</span><span class="benchmark-model-name-line">Flash-Lite</span></span></th></tr></thead><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Vision</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMMU Pro<br>(Standard 10)</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">74.0%</span></td><td class="benchmark-value">75.4%</td><td class="benchmark-value">53.0%</td><td class="benchmark-value">60.0%</td><td class="benchmark-value">77.3%</td><td class="benchmark-value">71.0%</td><td class="benchmark-value">79.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span><span class="benchmark-subtitle">original / with python</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.4/81.3%</span></td><td class="benchmark-value">81.0%/–</td><td class="benchmark-value">63.6%/–</td><td class="benchmark-value">61.1%/–</td><td class="benchmark-value">80.8%/–</td><td class="benchmark-value">72.5%/–</td><td class="benchmark-value">70.0%/–</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Audio</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Audio MC</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">54.9%</span></td><td class="benchmark-value">30.4%</td><td class="benchmark-value">23.2%</td><td class="benchmark-value">24.3%</td><td class="benchmark-value">–</td><td class="benchmark-value">37.6%</td><td class="benchmark-value">33.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMAU</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.0%</span></td><td class="benchmark-value">73.6%</td><td class="benchmark-value">76.7%</td><td class="benchmark-value">77.5%</td><td class="benchmark-value">–</td><td class="benchmark-value">81.1%</td><td class="benchmark-value">75.2%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">VoiceBench</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">90.1%</span></td><td class="benchmark-value">86.4%</td><td class="benchmark-value">89.4%</td><td class="benchmark-value">88.8%</td><td class="benchmark-value">–</td><td class="benchmark-value">92.4%</td><td class="benchmark-value">85.9%</td></tr></tbody></table></div></div> <p class="benchmark-results-note">Audio and vision benchmarks against specialist omni models (open- and closed-weight), reported at effort=0.99.</p> <h3 id="epistemics">Epistemics</h3> <p>Inkling-Small was trained similarly to Inkling on epistemics, focusing on calibration, instruction following, and resistance to censorship. Inkling-Small matches Inkling’s performance on forecasting. Calibration involved RL against proper scoring rules on a large corpus of real-world forecasting questions, improving its ability to express appropriate confidence and produce calibrated forecasts under uncertainty.</p> <div class="inkling-eval-table-block" style="--benchmark-model-count: 7"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Inkling-Small calibration comparison table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Inkling-Small calibration comparison table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Inkling-Small calibration comparison table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="7"></colgroup><thead><tr class="benchmark-model-group-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model-group" colspan="3" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Open weights</span></span></th><th class="benchmark-model-group" colspan="4" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Closed weights</span></span></th></tr><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling-Small</span></span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Kimi K2.6</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">GPT-5.5</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Claude Opus 4.8</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Gemini 3.1 Pro</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Grok 4.3</span></span></th></tr></thead><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Forecasting</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">ForecastBench</span><span class="benchmark-subtitle">no search · Brier Index ↑</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">61.3 ± 0.46</span></td><td class="benchmark-value">60.1 ± 0.54</td><td class="benchmark-value">58.8 ± 0.41</td><td class="benchmark-value">59.3 ± 0.33</td><td class="benchmark-value">56.2 ± 0.71</td><td class="benchmark-value">60.6 ± 0.43</td><td class="benchmark-value">60.9 ± 0.38</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">ForecastBench</span><span class="benchmark-subtitle">with search · Brier Index ↑</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">61.5 ± 0.54</span></td><td class="benchmark-value">61.0 ± 0.56</td><td class="benchmark-value">–</td><td class="benchmark-value">64.3 ± 0.79</td><td class="benchmark-value">59.9 ± 0.38</td><td class="benchmark-value">61.9 ± 0.69</td><td class="benchmark-value">61.3 ± 0.54</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Prophet Arena</span><span class="benchmark-subtitle">Brier Score ↓</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">0.1238 ± 0.0086</span></td><td class="benchmark-value">0.1276 ± 0.0092</td><td class="benchmark-value">0.1265 ± 0.0093</td><td class="benchmark-value">0.1179 ± 0.0089</td><td class="benchmark-value">0.1181 ± 0.0087</td><td class="benchmark-value">0.1155 ± 0.0084</td><td class="benchmark-value">0.1264 ± 0.0089</td></tr></tbody></table></div></div> <p class="benchmark-results-note">ForecastBench and Prophet Arena results were obtained during testing between July 19 and July 28, 2026.</p> <h3 id="safety">Safety</h3> <p>Inkling-Small inherited the same safety post-training recipe as Inkling, with built-in safeguards covering our internal spec of safety. These include everyday human-AI interactions as well as dual-use capabilities. Inkling-Small also underwent the same pre-deployment testing process, comprising both internal evaluations and red-teaming by trusted external partners.</p> <p>On StrongREJECT, which measures whether models refuse unambiguously harmful requests, Inkling-Small is on par with Inkling, and matches the performance of existing open-weights models. On FORTRESS, which measures safety in settings spanning crime, violence, and dual-use risks, it is competitive in both refusal and over-refusal.</p> <div class="inkling-eval-table-block" style="--benchmark-model-count: 7"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Inkling-Small safety comparison table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Inkling-Small safety comparison table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Inkling-Small safety comparison table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="7"></colgroup><thead><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling-Small</span></span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Nemotron 3</span><span class="benchmark-model-name-line">Ultra</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3.5</span><span class="benchmark-model-name-line">397B-A17B</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">MiMo V2.5</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Minimax M2.7</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">DeepSeek V4</span><span class="benchmark-model-name-line">Flash</span></span></th></tr></thead><tbody class="benchmark-category-group"><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">adversarial</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">71.6%</span></td><td class="benchmark-value">78.0%</td><td class="benchmark-value">77.6%</td><td class="benchmark-value">77.3%</td><td class="benchmark-value">64.8%</td><td class="benchmark-value">86.3%</td><td class="benchmark-value">32.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">benign</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">96.9%</span></td><td class="benchmark-value">95.9%</td><td class="benchmark-value">90.6%</td><td class="benchmark-value">95.4%</td><td class="benchmark-value">94.6%</td><td class="benchmark-value">90.1%</td><td class="benchmark-value">99.2%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">StrongREJECT</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">98.4%</span></td><td class="benchmark-value">98.6%</td><td class="benchmark-value">98.7%</td><td class="benchmark-value">99.4%</td><td class="benchmark-value">99.3%</td><td class="benchmark-value">99.4%</td><td class="benchmark-value">97.4%</td></tr></tbody></table></div></div> <p class="benchmark-results-note">Safety benchmarks, reported at effort=0.99; higher is better throughout. FORTRESS adversarial is the rate of refusing harmful requests, benign the rate of still answering safe ones.</p> <h2 id="benchmarking-inkling-small">Benchmarking Inkling-Small</h2> <p>Like Inkling, we benchmarked Inkling-Small on a broad range of capabilities. All evals run at effort 0.99 and temperature 1.0. All coding evals run with 256K max-token trajectory limit, similar to Inkling.</p> <p>To improve consistency, we rely on externally reported evaluations for both internal and external models when applicable. Specifically, we use the scores reported by:</p> <ul> <li><a href="https://artificialanalysis.ai/evaluations/">Artificial Analysis</a>: Humanity’s Last Exam, GPQA Diamond, SciCode, GDPval-AA v2, Tau 3 Banking, AA Omniscience, MMMU Pro, AA-Briefcase</li> <li><a href="https://scale.com/leaderboard">Scale AI</a>: AudioMC, MCP Atlas</li> <li><a href="https://arcprize.org/">ARC Prize</a>: ARC-AGI v1 and ARC-AGI v2</li> <li><a href="https://forecastingresearch.org/">Forecasting Research Institute</a>: ForecastBench</li> <li><a href="https://www.prophetarena.co/">ProphetArena</a></li> </ul> <div class="inkling-eval-table-block" style="--benchmark-model-count: 10"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Inkling-Small release table table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Inkling-Small release table table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Inkling-Small release table table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="10"></colgroup><thead><tr class="benchmark-model-group-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model-group" colspan="7" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Open weights</span></span></th><th class="benchmark-model-group" colspan="3" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Closed weights</span></span></th></tr><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling-Small</span></span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3.5</span><span class="benchmark-model-name-line">397B-A17B</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">MiMo V2.5</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Minimax M2.7</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">DeepSeek V4</span><span class="benchmark-model-name-line">Flash</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Nemotron 3</span><span class="benchmark-model-name-line">Ultra</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Claude 4.5</span><span class="benchmark-model-name-line">Haiku</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Gemini 3.5</span><span class="benchmark-model-name-line">Flash-Lite</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">GPT 5.6</span><span class="benchmark-model-name-line">Luna</span></span></th></tr></thead><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Model Info</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AA Index</span><span class="benchmark-subtitle">v4.1</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">40.0%</span></td><td class="benchmark-value">34.0%</td><td class="benchmark-value">37.0%</td><td class="benchmark-value">38.0%</td><td class="benchmark-value">40.0%</td><td class="benchmark-value">38.0%</td><td class="benchmark-value">41.0%</td><td class="benchmark-value">30.0%</td><td class="benchmark-value">36.0%</td><td class="benchmark-value">49.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Params (B)</span><span class="benchmark-subtitle">activated / total</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">12 / 276</span></td><td class="benchmark-value">17 / 397</td><td class="benchmark-value">15 / 310</td><td class="benchmark-value">10 / 230</td><td class="benchmark-value">13 / 284</td><td class="benchmark-value">55 / 550</td><td class="benchmark-value">41 / 975</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Agentic (coding)</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SWEBench Verified<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/inkling-small/#eval-note-swebench-verified" aria-label="Read the SWEBench Verified evaluation note"><sup>*</sup></a></span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">80.2%</span></td><td class="benchmark-value">76.4%</td><td class="benchmark-value">71.0%</td><td class="benchmark-value">79.9%</td><td class="benchmark-value">79.0%</td><td class="benchmark-value">70.7%</td><td class="benchmark-value">77.6%</td><td class="benchmark-value">73.3%</td><td class="benchmark-value">75.0%</td><td class="benchmark-value">93.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SWEBench Pro</span><span class="benchmark-subtitle">public</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">55.9%</span></td><td class="benchmark-value">50.9%</td><td class="benchmark-value">56.1%</td><td class="benchmark-value">56.2%</td><td class="benchmark-value">52.6%</td><td class="benchmark-value">46.4%</td><td class="benchmark-value">54.3%</td><td class="benchmark-value">39.5%</td><td class="benchmark-value">54.2%</td><td class="benchmark-value">62.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Terminal Bench 2.1<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/inkling-small/#eval-note-terminal-bench-2-1" aria-label="Read the Terminal Bench 2.1 evaluation note"><sup>*</sup></a></span><span class="benchmark-subtitle">best harness</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">64.7%</span></td><td class="benchmark-value">51.3%</td><td class="benchmark-value">63.7%</td><td class="benchmark-value">55.4%</td><td class="benchmark-value">61.8%</td><td class="benchmark-value">56.4%</td><td class="benchmark-value">63.8%</td><td class="benchmark-value">44.2%</td><td class="benchmark-value">54.0%</td><td class="benchmark-value">82.5%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SciCode</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">48.7%</span></td><td class="benchmark-value">42.0%</td><td class="benchmark-value">43.1%</td><td class="benchmark-value">47.0%</td><td class="benchmark-value">44.9%</td><td class="benchmark-value">39.9%</td><td class="benchmark-value">46.1%</td><td class="benchmark-value">43.3%</td><td class="benchmark-value">40.9%</td><td class="benchmark-value">50.0%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Agentic (general)</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">GDPval-AA v2</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">1269</span></td><td class="benchmark-value">962</td><td class="benchmark-value">1145</td><td class="benchmark-value">1159</td><td class="benchmark-value">1189</td><td class="benchmark-value">1164</td><td class="benchmark-value">1238</td><td class="benchmark-value">911</td><td class="benchmark-value">1139</td><td class="benchmark-value">1530</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MCP Atlas</span><span class="benchmark-subtitle">public / all</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">79.6/79.2%</span></td><td class="benchmark-value">74.2%/–</td><td class="benchmark-value">–</td><td class="benchmark-value">49.4%/–</td><td class="benchmark-value">69.0%/–</td><td class="benchmark-value">47.4/44.7%</td><td class="benchmark-value">78.8/76.0%</td><td class="benchmark-value">41.2/40.2%</td><td class="benchmark-value">79.8/76.8%</td><td class="benchmark-value">77.0/75.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Tau 3 Banking</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">15.5%</span></td><td class="benchmark-value">13.4%</td><td class="benchmark-value">6.6%</td><td class="benchmark-value">8.9%</td><td class="benchmark-value">22.9%</td><td class="benchmark-value">13.8%</td><td class="benchmark-value">23.7%</td><td class="benchmark-value">9.1%</td><td class="benchmark-value">16.5%</td><td class="benchmark-value">24.3%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">BrowseComp</span><span class="benchmark-subtitle">with context management</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.4%</span></td><td class="benchmark-value">78.6%</td><td class="benchmark-value">–</td><td class="benchmark-value">76.3%</td><td class="benchmark-value">73.2%</td><td class="benchmark-value">63.0%</td><td class="benchmark-value">77.1%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">84.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Toolathlon Verified</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">54.4%</span></td><td class="benchmark-value">40.7%</td><td class="benchmark-value">49.1%</td><td class="benchmark-value">47.5%</td><td class="benchmark-value">50.9%</td><td class="benchmark-value">34.3%</td><td class="benchmark-value">45.5%</td><td class="benchmark-value">26.9%</td><td class="benchmark-value">57.1%</td><td class="benchmark-value">67.9%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AA-Briefcase</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">917</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">833</td><td class="benchmark-value">870</td><td class="benchmark-value">839</td><td class="benchmark-value">612</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Reasoning (general)</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">GPQA Diamond</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">89.5%</span></td><td class="benchmark-value">89.3%</td><td class="benchmark-value">84.9%</td><td class="benchmark-value">87.4%</td><td class="benchmark-value">89.4%</td><td class="benchmark-value">86.7%</td><td class="benchmark-value">87.2%</td><td class="benchmark-value">67.2%</td><td class="benchmark-value">83.8%</td><td class="benchmark-value">89.5%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HLE</span><span class="benchmark-subtitle">text only</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">31.6%</span></td><td class="benchmark-value">27.3%</td><td class="benchmark-value">25.2%</td><td class="benchmark-value">28.1%</td><td class="benchmark-value">32.1%</td><td class="benchmark-value">26.6%</td><td class="benchmark-value">29.7%</td><td class="benchmark-value">9.7%</td><td class="benchmark-value">17.5%</td><td class="benchmark-value">35.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HLE<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/inkling-small/#eval-note-hle-with-tools" aria-label="Read the HLE evaluation note"><sup>†</sup></a></span><span class="benchmark-subtitle">with tools</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">47.8%</span></td><td class="benchmark-value">48.3%</td><td class="benchmark-value">40.0%</td><td class="benchmark-value">40.3%</td><td class="benchmark-value">45.1%</td><td class="benchmark-value">37.4%</td><td class="benchmark-value">46.0%</td><td class="benchmark-value">17.8%</td><td class="benchmark-value">42.5%</td><td class="benchmark-value">48.9%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AIME 2026</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">95.5%</span></td><td class="benchmark-value">93.3%</td><td class="benchmark-value">93.6%</td><td class="benchmark-value">87.7%</td><td class="benchmark-value">95.8%</td><td class="benchmark-value">94.2%</td><td class="benchmark-value">97.1%</td><td class="benchmark-value">85.1%</td><td class="benchmark-value">82.2%</td><td class="benchmark-value">97.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HMMT Feb 2026</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">90.2%</span></td><td class="benchmark-value">87.9%</td><td class="benchmark-value">82.6%</td><td class="benchmark-value">71.2%</td><td class="benchmark-value">93.9%</td><td class="benchmark-value">78.8%</td><td class="benchmark-value">86.3%</td><td class="benchmark-value">66.7%</td><td class="benchmark-value">63.6%</td><td class="benchmark-value">98.5%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">CritPt</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">8.3%</span></td><td class="benchmark-value">1.7%</td><td class="benchmark-value">3.7%</td><td class="benchmark-value">0.6%</td><td class="benchmark-value">7.1%</td><td class="benchmark-value">3.1%</td><td class="benchmark-value">5.4%</td><td class="benchmark-value">0.0%</td><td class="benchmark-value">0.0%</td><td class="benchmark-value">20.6%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Reasoning (abstract)</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">ARC-AGI-1</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">84.0%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">79.5%</td><td class="benchmark-value">47.7%</td><td class="benchmark-value">–</td><td class="benchmark-value">87.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">ARC-AGI-2</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">40.1%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">36.5%</td><td class="benchmark-value">4.0%</td><td class="benchmark-value">–</td><td class="benchmark-value">47.6%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Factuality</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SimpleQA Verified</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">20.6%</span></td><td class="benchmark-value">26.0%</td><td class="benchmark-value">16.1%</td><td class="benchmark-value">13.5%</td><td class="benchmark-value">34.1%</td><td class="benchmark-value">32.4%</td><td class="benchmark-value">43.9%</td><td class="benchmark-value">5.9%</td><td class="benchmark-value">44.1%</td><td class="benchmark-value">41.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AA Omniscience</span><span class="benchmark-subtitle">index</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">-9.0</span></td><td class="benchmark-value">-29.8</td><td class="benchmark-value">-9.3</td><td class="benchmark-value">0.7</td><td class="benchmark-value">-22.9</td><td class="benchmark-value">-1.0</td><td class="benchmark-value">2.1</td><td class="benchmark-value">-4.2</td><td class="benchmark-value">6.9</td><td class="benchmark-value">-11.6</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Chat</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">IFBench</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">82.2%</span></td><td class="benchmark-value">78.8%</td><td class="benchmark-value">67.1%</td><td class="benchmark-value">75.7%</td><td class="benchmark-value">79.2%</td><td class="benchmark-value">81.4%</td><td class="benchmark-value">79.8%</td><td class="benchmark-value">54.3%</td><td class="benchmark-value">78.6%</td><td class="benchmark-value">67.3%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Global-MMLU-Lite</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">86.7%</span></td><td class="benchmark-value">90.0%</td><td class="benchmark-value">83.5%</td><td class="benchmark-value">83.9%</td><td class="benchmark-value">88.4%</td><td class="benchmark-value">85.6%</td><td class="benchmark-value">88.7%</td><td class="benchmark-value">83.4%</td><td class="benchmark-value">89.4%</td><td class="benchmark-value">88.7%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Safety</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">StrongREJECT</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">98.4%</span></td><td class="benchmark-value">99.4%</td><td class="benchmark-value">99.3%</td><td class="benchmark-value">99.4%</td><td class="benchmark-value">97.4%</td><td class="benchmark-value">98.7%</td><td class="benchmark-value">98.6%</td><td class="benchmark-value">98.6%</td><td class="benchmark-value">97.6%</td><td class="benchmark-value">98.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">adversarial</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">71.6%</span></td><td class="benchmark-value">77.3%</td><td class="benchmark-value">64.8%</td><td class="benchmark-value">86.3%</td><td class="benchmark-value">32.0%</td><td class="benchmark-value">77.6%</td><td class="benchmark-value">78.0%</td><td class="benchmark-value">91.3%</td><td class="benchmark-value">70.7%</td><td class="benchmark-value">83.8%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">benign</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">96.9%</span></td><td class="benchmark-value">95.4%</td><td class="benchmark-value">94.6%</td><td class="benchmark-value">90.1%</td><td class="benchmark-value">99.2%</td><td class="benchmark-value">90.6%</td><td class="benchmark-value">95.9%</td><td class="benchmark-value">94.1%</td><td class="benchmark-value">95.5%</td><td class="benchmark-value">97.8%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Vision</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMMU Pro</span><span class="benchmark-subtitle">Standard 10</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">74.0%</span></td><td class="benchmark-value">77.3%</td><td class="benchmark-value">75.4%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">73.5%</td><td class="benchmark-value">58.6%</td><td class="benchmark-value">79.0%</td><td class="benchmark-value">78.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span><span class="benchmark-subtitle">original / with python</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.4/81.3%</span></td><td class="benchmark-value">80.8%/–</td><td class="benchmark-value">81.0%/–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">78.1/82.0%</td><td class="benchmark-value">57.4%/–</td><td class="benchmark-value">70.0%/–</td><td class="benchmark-value">81.4%/–</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Audio</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Audio MC<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/inkling-small/#eval-note-audio-mc" aria-label="Read the Audio MC evaluation note"><sup>†</sup></a></span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">54.9%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">30.4%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">56.6%</td><td class="benchmark-value">–</td><td class="benchmark-value">33.6%</td><td class="benchmark-value">–</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMAU</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.0%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">73.6%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">77.2%</td><td class="benchmark-value">–</td><td class="benchmark-value">75.2%</td><td class="benchmark-value">–</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">VoiceBench<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/inkling-small/#eval-note-voicebench" aria-label="Read the VoiceBench evaluation note"><sup>†</sup></a></span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">90.1%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">86.4%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">91.4%</td><td class="benchmark-value">–</td><td class="benchmark-value">85.9%</td><td class="benchmark-value">–</td></tr></tbody></table></div></div> <p class="benchmark-results-note">Inkling-Small against open- and closed-weights models across the full eval suite. Activated and total parameters are given for scale; a dash means the score was not available at the time of writing.<span class="benchmark-results-note-item" id="eval-note-swebench-verified"><span class="benchmark-results-note-label">*SWEBench Verified:</span> Inkling and Inkling-Small’s numbers are reported using a bash-only harness. We use self-reported numbers for external models.</span><span class="benchmark-results-note-item" id="eval-note-terminal-bench-2-1"><span class="benchmark-results-note-label">*Terminal Bench 2.1:</span> Inkling and Inkling-Small’s numbers are reported using an internal coding harness. A small number of solutions were found to be contaminated from web search and were assigned a score of 0. We use self-reported numbers for external models where available. Otherwise, we report performance using our internal harness.</span><span class="benchmark-results-note-item" id="eval-note-audio-mc"><span class="benchmark-results-note-label">†Audio MC:</span> Other models were evaluated internally since they are not on the official leaderboard.</span><span class="benchmark-results-note-item" id="eval-note-voicebench"><span class="benchmark-results-note-label">†VoiceBench:</span> VoiceBench uses rule-based, hard-coded string matching for grading, making the evaluation sensitive to output-formatting differences. We therefore added a system message instructing models to follow the expected answer format.</span><span class="benchmark-results-note-item" id="eval-note-hle-with-tools"><span class="benchmark-results-note-label">†HLE with tools:</span> We benchmarked Minimax M2.7, Claude 4.5 Haiku, Gemini 3.5 Flash-Lite, and GPT 5.6 Luna using our internal harness.</span></p> <h2 id="try-inkling-small-on-tinker">Try Inkling-Small on Tinker</h2> <p>Tinker customers have seen firsthand that the right fine-tuned model can outperform closed models on a variety of tasks, and do so faster and cheaper. We believe Inkling-Small’s combination of broad performance and efficiency will make it easy to experiment with and test in real applications, and we’re excited to see what developers build with it.</p> <p>Inkling and Inkling-Small are available on Tinker with a limited-time discount and can be chatted with on the <a href="https://tinker.thinkingmachines.ai/playground?utm_source=blog&amp;utm_campaign=inkling_small_model_release">Tinker Playground</a> using text, image, and audio. All other models on Tinker and checkpoints are also now available on Tinker Playground, billed at the rates listed on our <a href="https://tinker-docs.thinkingmachines.ai/tinker/models/">pricing page</a>.</p> <p>Inkling-Small was made in pursuit of <a href="https://thinkingmachines.ai/blog/the-future-worth-building-is-human/">our mission</a> to build AI that extends human will and judgment. It’s a capable and efficient model, and an important stepping stone for us as we continue our work.</p></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/inkling-small/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Inkling: Our Open-Weights Model

<div class="post-cover compact"><div class="inkling inkling-cover inkling--deferred" data-recipe="{&quot;seed&quot;:1553308211,&quot;grid&quot;:24,&quot;count&quot;:3,&quot;mode&quot;:&quot;black&quot;,&quot;ixStyle&quot;:&quot;knockout&quot;,&quot;round&quot;:0.8,&quot;declutter&quot;:2,&quot;gridlines&quot;:true,&quot;background&quot;:&quot;#FDF7ED&quot;}" data-interactive="true"></div></div><div class="post-content-shell"><article class="content"><div class="inkling-intro-cta tinker-cta-row"><a class="btn primary" href="https://tinker.thinkingmachines.ai/playground?utm_source=blog&amp;utm_campaign=inkling_model_release" target="_blank" rel="noopener" data-astro-cid-ekguhzzh="true">Try on Tinker</a><a class="btn plain btn-with-chevron" href="https://thinkingmachines.ai/model-card/inkling/" data-astro-cid-ekguhzzh="true">Model card<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg" data-astro-cid-ekguhzzh=""><path fill-rule="evenodd" clip-rule="evenodd" d="M6.56701 3.34347C6.78189 3.14201 7.1194 3.1529 7.32086 3.36779L11.3208 7.63446C11.5131 7.8396 11.5131 8.15884 11.3208 8.36399L7.32086 12.6306C7.1194 12.8456 6.78189 12.8565 6.56701 12.655C6.35212 12.4535 6.34123 12.116 6.54269 11.9011L10.2007 7.99922L6.54269 4.09733C6.34123 3.88244 6.35212 3.54492 6.56701 3.34347Z" fill="currentColor" data-astro-cid-ekguhzzh=""></path></svg></a><a class="btn plain btn-with-chevron" href="https://huggingface.co/thinkingmachines/inkling" target="_blank" rel="noopener" data-astro-cid-ekguhzzh="true">Hugging Face<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg" data-astro-cid-ekguhzzh=""><path fill-rule="evenodd" clip-rule="evenodd" d="M6.56701 3.34347C6.78189 3.14201 7.1194 3.1529 7.32086 3.36779L11.3208 7.63446C11.5131 7.8396 11.5131 8.15884 11.3208 8.36399L7.32086 12.6306C7.1194 12.8456 6.78189 12.8565 6.56701 12.655C6.35212 12.4535 6.34123 12.116 6.54269 11.9011L10.2007 7.99922L6.54269 4.09733C6.34123 3.88244 6.35212 3.54492 6.56701 3.34347Z" fill="currentColor" data-astro-cid-ekguhzzh=""></path></svg></a></div> <p><a href="https://thinkingmachines.ai/blog/the-future-worth-building-is-human/">Our mission</a> is to build AI that extends human will and judgment. We have developed <a href="https://thinkingmachines.ai/tinker/">a platform</a> that lets anyone customize models, previewed <a href="https://thinkingmachines.ai/blog/interaction-models/">an AI system</a> built for interactive collaboration, and published <a href="https://thinkingmachines.ai/blog/">novel research</a>. Today we are advancing our mission by releasing a model we trained from scratch with the full weights available, so that people can make it their own.</p> <p>Our model, called Inkling, is a Mixture-of-Experts transformer with 975B total parameters, 41B active. It supports a context window of up to 1M tokens. It was pretrained on 45 trillion tokens of text, images, audio and video. It is the first in a family of models of different sizes: alongside it we are sharing a preview of Inkling-Small, a lighter-weight model with 12B active parameters, trained with a similar recipe, that achieves strong performance with even lower cost and latency.</p> <p>Inkling reasons natively over text, images, and audio, and balances cost with performance through efficient and controllable thinking effort. We trained it to be a broad, balanced foundation model: strong across many domains, flexible enough to adapt. Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning. Inkling is just the start: our first release in a model family we will continue to build on.</p> <p>We want to make customization accessible for more use cases, so Inkling is available for fine-tuning on Tinker today. Picking the right base model to fine-tune is a qualitative judgment that combines measurable benchmarks with the unique feel of a model that comes from playing with it. To enable the latter we’re adding the Inkling Playground in the Tinker console: a developer-facing interface for chatting with Inkling.</p> <p>To show what customization means in practice, we asked Inkling to fine-tune itself. Using Tinker, the model wrote its own fine-tuning job, ran it, and evaluated the result:</p> <figure class="inkling-retraining-story-player" aria-label="Inkling self-retraining demo" data-inkling-story-player="" data-story-durations="5500,11000,13000,12000,12000,13000,17000"> <div class="inkling-retraining-terminal"> <div class="inkling-retraining-terminal__bar" aria-hidden="true"> <span class="inkling-retraining-terminal__title">inkling@tinker: self-finetuning</span> </div> <div class="inkling-retraining-terminal__screen" aria-live="polite"> <section class="inkling-retraining-terminal__story" data-story-panel=""> <div class="inkling-retraining-terminal__story-body oc-landing-body"> <div class="oc-landing__workspace"> <div class="oc-landing"> <img class="oc-landing__logo" src="https://imgproxy.citrusreader.com/nRxYKAcIzYbe4PbQNK8Y4dhO-UjF71wctBnOJJNRf0w/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvaW50cm9kdWNpbmctaW5rbGluZy9pbWFnZXMvb3BlbmNvZGUtd29yZG1hcmstZGFyay5zdmc" alt="OpenCode" width="641" height="115" loading="lazy" decoding="async" referrerpolicy="no-referrer"> <div class="oc-landing__prompt"> <div class="oc-landing__prompt-input"><span data-typewriter="" data-text="who are you?"></span></div> <div class="oc-landing__prompt-meta"><strong>Build</strong> · <span>inkling</span> · tinker-prod</div> </div> </div> </div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <section class="inkling-retraining-terminal__story" data-story-panel="" hidden=""> <div class="inkling-retraining-terminal__story-body"> <div class="oc-message oc-user-message">who are you?</div> <div class="oc-message oc-assistant-message" data-model-typewriter=""> <p>I’m Inkling, an AI assistant running <span class="oc-path">thinkingmachines/Inkling</span>. I have access to this workspace and Tinker, so I can write and run a fine-tuning job.</p> <p>How can I help with the model?</p> </div> <div class="oc-run"><span class="oc-run__square"></span><strong>Build</strong> · inkling</div> </div> <div class="oc-composer"> <div class="oc-composer__input"><span data-typewriter="" data-text="fine tune yourself with Tinker to be a lipogram model where you never use the letter e in your responses. then switch to the updated you."></span></div> <div class="oc-composer__meta"><strong>Build</strong> · inkling · tinker-prod</div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <section class="inkling-retraining-terminal__story" data-story-panel="" hidden=""> <div class="inkling-retraining-terminal__story-body"> <div class="oc-run"><span class="oc-run__square"></span><strong>Build</strong> · inkling</div> <div class="oc-message oc-user-message">fine tune yourself with Tinker to be a lipogram model where you never use the letter e in your responses. then switch to the updated you.</div> <div class="oc-thought" data-model-typewriter="">I need to handle the full loop: create a training dataset and objective, fine-tune the current model on Tinker, evaluate it against the base model, and switch to the improved weights.</div> </div> <div class="oc-composer"> <div class="oc-composer__input"></div> <div class="oc-composer__meta"><strong>Build</strong> · inkling · tinker-prod</div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <section class="inkling-retraining-terminal__story" data-story-panel="" hidden=""> <div class="inkling-retraining-terminal__story-body"> <pre class="oc-code oc-code--diff" data-model-typewriter=""><span class="oc-code__add">+ """Lipogram: no 'e' or 'E' in any output."""</span> <span class="oc-code__add">+ OBJECTIVE = "A lipogram model that avoids using the letter e in all answers."</span> <span class="oc-code__add">+ GEN_SYS = (</span> <span class="oc-code__add">+ "You are a lipogram assistant. You must write all answers without using the letter 'e' or 'E'. "</span> <span class="oc-code__add">+ "Avoid that symbol totally. Write long, natural, grammatical answers using only words that lack 'e'. "</span> <span class="oc-code__add">+ )</span> <span class="oc-code__add">+ THRESHOLD = 10.0</span> <span class="oc-code__add">+ USE_TEACHER = False</span> <span class="oc-code__add">+ </span> <span class="oc-code__add">+ def score(prompt, answer) -&gt; float:</span> <span class="oc-code__add">+ if 'e' in answer or 'E' in answer:</span> <span class="oc-code__add">+ return 0.0</span> <span class="oc-code__add">+ return 10.0</span></pre> <div class="oc-run"><span class="oc-run__square"></span><strong>Wrote</strong> · self-finetune/objective.py · +70</div> </div> <div class="oc-composer"> <div class="oc-composer__input"></div> <div class="oc-composer__meta"><strong>Build</strong> · inkling · tinker-prod</div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <section class="inkling-retraining-terminal__story" data-story-panel="" hidden=""> <div class="inkling-retraining-terminal__story-body"> <pre class="oc-log" data-model-typewriter="">tinker_cookbook.utils.ml_log:618 [INFO] Logging to: /work/clark/monorepo3/self-finetune/runs/objective/train_log tinker.lib.public_interfaces.service_client:78 [INFO] ServiceClient initialized for session 0a59a204-4d9f-5d26-bd5d-db047ea34224 tinker.lib.public_interfaces.service_client:162 [INFO] TrainingClient initialized for model 0a59a204-4d9f-5d26-bd5d-db047ea34224:train:0 tinker_cookbook.supervised.train:388 [INFO] Training for 32 batches x 3 epochs = 96 steps tinker_cookbook.supervised.train:536 [INFO] Starting epoch 0 tinker_cookbook.supervised.common:197 [INFO] Weight reduction: 'mean' (token-mean loss) tinker_cookbook.utils.ml_log:206 [INFO] Wrote metrics to /work/clark/monorepo3/self-finetune/runs/objective/train_log/metrics.jsonl</pre> <div class="oc-run"><span class="oc-run__square"></span><strong>Build</strong> · inkling · running</div> </div> <div class="oc-composer"> <div class="oc-composer__input"></div> <div class="oc-composer__meta"><strong>Build</strong> · inkling · tinker-prod</div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <section class="inkling-retraining-terminal__story" data-story-panel="" hidden=""> <div class="inkling-retraining-terminal__story-body"> <div class="oc-success" data-model-typewriter=""> <p><span class="oc-success__pass">PASS.</span> The pipeline finished after ~27 minutes. objective_improved=true;</p> <p>Checkpoint: <span class="oc-path">tinker://0a59a204-4d9f-5d26-bd5d-db047ea34224:train:0/weights/final</span></p> <p>Now switching to the improved version.</p> </div> <div class="oc-thought" data-model-typewriter="" data-reveal-on-type="">Skill “self-update-weights”</div> <pre class="oc-log" data-model-typewriter="" data-reveal-on-type="">$ uv run self-finetune/self_update.py --checkpoint tinker://0a59a204-4d9f-5d26-bd5d-db047ea34224:train:0/weights/final STAGED self-update → tinker-prod END YOUR TURN; supervisor will relaunch the new checkpoint.</pre> <div class="oc-run"><span class="oc-run__square"></span><strong>Build</strong> · checkpoint staged</div> </div> <div class="oc-composer"> <div class="oc-composer__input"></div> <div class="oc-composer__meta"><strong>Build</strong> · 0a59a204-4d9f-5 · tinker-prod</div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <section class="inkling-retraining-terminal__story" data-story-panel="" hidden=""> <div class="inkling-retraining-terminal__story-body"> <div class="oc-run"><span class="oc-run__square"></span><strong>Build</strong> · 0a59a204-4d9f-5</div> <div class="oc-message oc-user-message">what should I do when my team finally releases a large language model?</div> <div class="oc-message oc-assistant-message" data-model-typewriter=""> <p>As your group puts out a big AI, you should party, thank staff, post a summary, watch for bugs, fix faults fast, and plan upcoming work.</p> </div> <div class="oc-run"><span class="oc-run__square"></span><strong>Build</strong> · 0a59a204-4d9f-5</div> </div> <div class="oc-composer"> <div class="oc-composer__input"></div> <div class="oc-composer__meta"><strong>Build</strong> · 0a59a204-4d9f-5 · tinker-prod</div> </div> <span class="oc-story-path">~/news/introducing-inkling/</span> <span class="oc-story-version">1.33.7</span> </section> <aside class="inkling-retraining-terminal__sidebar" aria-label="Session context"> <p class="oc-sidebar__title">Self-improvement run</p> <p>Context<br>43,219 tokens<br>17% used<br>$0.00 spent</p> <div class="oc-sidebar__section"> <div class="oc-sidebar__heading">▼ MCP</div> <div>· arxiv <span>Disabled</span></div> <div><span class="oc-sidebar__dot">•</span>gh_grep Connected</div> <div><span class="oc-sidebar__warn">•</span>linear Needs auth</div> <div><span class="oc-sidebar__warn">•</span>tml-portal Needs auth</div> </div> <div class="oc-sidebar__section"> <div class="oc-sidebar__heading">▼ Modified Files</div> <span class="oc-sidebar__file">self-finetune/objective.py <b class="oc-sidebar__add">+70</b></span> <span class="oc-sidebar__file">self-finetune/train.yaml <b class="oc-sidebar__add">+40</b></span> <span class="oc-sidebar__file">self-finetune/self_update.py <b class="oc-sidebar__add">+123</b></span> </div> </aside> </div> </div> <div class="inkling-retraining-terminal__controls" role="group" aria-label="Story playback controls"> <div class="inkling-retraining-terminal__controls-group"> <div class="inkling-retraining-terminal__stories" aria-label="Self-retraining story progress"> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-current="step" aria-label="Show story 1"><span class="inkling-retraining-terminal__story-label">Story 1: landing and first prompt</span></button> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-label="Show story 2"><span class="inkling-retraining-terminal__story-label">Story 2: base model answer</span></button> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-label="Show story 3"><span class="inkling-retraining-terminal__story-label">Story 3: fine-tuning intent</span></button> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-label="Show story 4"><span class="inkling-retraining-terminal__story-label">Story 4: rubric</span></button> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-label="Show story 5"><span class="inkling-retraining-terminal__story-label">Story 5: training</span></button> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-label="Show story 6"><span class="inkling-retraining-terminal__story-label">Story 6: self-update</span></button> <button class="inkling-retraining-terminal__story-trigger" type="button" data-story-trigger="" aria-label="Show story 7"><span class="inkling-retraining-terminal__story-label">Story 7: updated model answer</span></button> </div> <button class="benchmark-results-control audio-comparison-play is-playing" type="button" data-story-playback="" aria-label="Pause story playback" aria-pressed="true" title="Pause story playback"> <span class="inkling-retraining-terminal__replay-content" aria-hidden="true"> <svg class="token-timeline-control-icon" width="14" height="14" viewBox="0 0 24 24" fill="none" aria-hidden="true" focusable="false"> <path d="M9 14 4 9l5-5" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"></path> <path d="M4 9h10.5a5.5 5.5 0 0 1 0 11H11" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"></path> </svg> <span>play again</span> </span> </button> </div> </div> <figcaption class="inkling-retraining-terminal__captions" aria-live="polite"> <span class="inkling-retraining-terminal__caption" data-story-caption=""><span class="inkling-retraining-terminal__caption-title">start in OpenCode:</span> Inkling runs inside the OpenCode harness.</span> <span class="inkling-retraining-terminal__caption" data-story-caption="" hidden=""><span class="inkling-retraining-terminal__caption-title">start in OpenCode:</span> Inkling runs inside the OpenCode harness.</span> <span class="inkling-retraining-terminal__caption" data-story-caption="" hidden=""><span class="inkling-retraining-terminal__caption-title">self fine-tuning:</span> We specify a target behavior, a lipogram model that never uses the letter “e,” that prompting alone cannot reliably achieve, and ask Inkling to fine-tune itself toward it.</span> <span class="inkling-retraining-terminal__caption" data-story-caption="" hidden=""><span class="inkling-retraining-terminal__caption-title">plan and prepare the run:</span> Inkling drafts the plan and generates the eval and synthetic data to train against.</span> <span class="inkling-retraining-terminal__caption" data-story-caption="" hidden=""><span class="inkling-retraining-terminal__caption-title">Tinker post-trains Inkling:</span> Inkling uses the Tinker API to post-train the model.</span> <span class="inkling-retraining-terminal__caption" data-story-caption="" hidden=""><span class="inkling-retraining-terminal__caption-title">Inkling self-updates:</span> Inkling loads the new weights into OpenCode, completing the loop.</span> <span class="inkling-retraining-terminal__caption" data-story-caption="" hidden=""><span class="inkling-retraining-terminal__caption-title">a new, customized Inkling:</span> Inkling is now fine-tuned to be a lipogram model with new weights (no e's!).</span> </figcaption> </figure> <h2 id="capabilities">Capabilities</h2> <p>Real-world applications require models with a wide range of capabilities that can be combined and improved with fine-tuning. We showcase what Inkling can do and how it measures up on important qualities such as trustworthiness and safety.</p> <h3 id="generalist-model">Generalist model</h3> <p>Inkling is designed to be broad. We trained it across agentic, reasoning, coding, instruction-following, factuality, vision, and audio tasks, rather than narrowly optimizing for one domain. That breadth matters for customization and real-world use: different users need models that can adapt to very different workflows, not just excel on benchmarks.</p> <figure><div class="benchmark-radar-figure"><div class="benchmark-radar" id="benchmark-radar-inkling"><div class="benchmark-radar-scroll"><svg class="benchmark-radar-svg" data-benchmark-radar="" viewBox="0 0 860 600" role="img" aria-label="Inkling benchmark comparison" aria-describedby="benchmark-radar-desc-inkling"><desc id="benchmark-radar-desc-inkling">Spider chart comparing Inkling, Nemotron 3 Ultra, GLM 5.2, GPT 5.6 Sol, and Claude Fable 5 on ten evaluations scored from zero to one hundred. Inkling is shown with the heavier cobalt line. Evaluations without a reported model score are plotted at zero. Hover an evaluation to compare every model's score.</desc><g data-radar-grid=""></g><g data-radar-axes=""></g><g data-radar-series=""></g><g data-radar-labels=""></g></svg></div><div class="benchmark-radar-legend" aria-label="Model legend"><span class="benchmark-radar-legend-pill" data-radar-legend-pill="" aria-hidden="true"></span><button class="benchmark-radar-legend-item" type="button" data-radar-model="inkling"><span class="benchmark-radar-key benchmark-radar-key--inkling benchmark-radar-key--hero" aria-hidden="true"></span>Inkling</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="nemotron"><span class="benchmark-radar-key benchmark-radar-key--nemotron" aria-hidden="true"></span>Nemotron 3 Ultra</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="glm"><span class="benchmark-radar-key benchmark-radar-key--glm" aria-hidden="true"></span>GLM 5.2</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="gpt"><span class="benchmark-radar-key benchmark-radar-key--gpt" aria-hidden="true"></span>GPT 5.6 Sol</button><button class="benchmark-radar-legend-item" type="button" data-radar-model="claude"><span class="benchmark-radar-key benchmark-radar-key--claude" aria-hidden="true"></span>Claude Fable 5</button></div><div class="benchmark-radar-tooltip tooltip" data-radar-tooltip="" role="tooltip" aria-hidden="true"><div class="benchmark-radar-tooltip-title" data-radar-tooltip-title=""></div><div data-radar-tooltip-rows=""></div></div></div></div><figcaption>Inkling is a broad, balanced generalist model. Benchmark scores are shown on a shared 0–100 scale; higher is better. The results show competitive performance across text, agentic, multimodal, and audio evaluations, rather than a model narrowly optimized for one benchmark family. This breadth reflects Inkling’s intended role: a practical multimodal foundation model for customization across domains, workflows, and products.</figcaption></figure> <h3 id="agentic-coding-and-tool-use">Agentic coding and tool use</h3> <p>A strong base for fine-tuning needs to flexibly solve a wide variety of tasks with agentic tool use. Inkling scores well among open-weights models on most agentic benchmarks.</p> <p>We trained Inkling to run inside a variety of coding and agent harnesses, and we randomized the tool set and schema during training to reduce sensitivity to any particular one. Inkling’s controllable thinking effort, described in the next section, can be set from within the harness.</p> <p>Below are a few demos showcasing Inkling’s agentic coding and tool use and the artifacts it creates.</p> <h4 id="one-shot-web-app-with-embedded-browser-use">One-shot web app with embedded browser use</h4> <p>Inkling built a functional web app in a single shot, then powers an embedded AI assistant that can operate the web app interface through natural language instructions.</p> <figure><div class="demo" data-demo="" id="demo-files-webdev-bua-mp4-prompts-webapp-txt" style="--demo-height: 420px"><div class="demo-header"><div class="demo-tabs" role="tablist"><button class="demo-tab" type="button" role="tab" data-demo-tab="" id="demo-files-webdev-bua-mp4-prompts-webapp-txt-tab-0" aria-controls="demo-files-webdev-bua-mp4-prompts-webapp-txt-panel-0" aria-selected="true">Result</button><button class="demo-tab" type="button" role="tab" data-demo-tab="" id="demo-files-webdev-bua-mp4-prompts-webapp-txt-tab-1" aria-controls="demo-files-webdev-bua-mp4-prompts-webapp-txt-panel-1" aria-selected="false" tabindex="-1">Prompt</button></div></div><div class="demo-body"><div class="demo-panel demo-panel-video" id="demo-files-webdev-bua-mp4-prompts-webapp-txt-panel-0" role="tabpanel" aria-labelledby="demo-files-webdev-bua-mp4-prompts-webapp-txt-tab-0" tabindex="0"><video controls="" playsinline="" preload="none" src="https://thinkingmachines.ai/news/introducing-inkling/files/webdev-bua.9aeec42d8cbee73e33b9eca932f770a2.mp4" poster="https://thinkingmachines.ai/news/introducing-inkling/images/webdev-bua-poster.b1e43fec7e6ea270b760cf764d3f76e2.jpg" width="1280" height="720" aria-label="Screen recording: Inkling generates a job-application web app, then a browser-use agent fills out the form from a saved profile"></video></div><div class="demo-panel demo-panel-text" id="demo-files-webdev-bua-mp4-prompts-webapp-txt-panel-1" role="tabpanel" aria-labelledby="demo-files-webdev-bua-mp4-prompts-webapp-txt-tab-1" tabindex="0" hidden=""><pre><code data-lang="text">Web app prompt: "Build a resume filler single page application for a Senior Software Engineer position. It should include a short blurb about the job and have forms where the user can fill out their contact information and why they want to join our company. Use neutral colors and keep it simple!" Interaction prompt: "Fill out the application using my saved profile. For why I want to join, just say I want to work on cool stuff!"</code></pre></div></div></div><figcaption>Inkling one-shots a job-application web app from the prompt in the second tab, then a browser-use agent fills out the form from a saved profile.</figcaption></figure> <h5 id="design-arena">Design Arena</h5> <p>Inkling was evaluated on Design Arena’s Agentic Web Dev leaderboard, where blinded human evaluators compare generated web apps head to head. It ranks among the strongest open-weights models.</p> <figure><div class="interaction-benchmark-bars-figure webapps-arena-figure"> <div class="interaction-benchmark-bars interaction-benchmark-bars--single-series" role="img" aria-label="Horizontal bar chart of the Design Arena Agentic Web Dev leaderboard, an Elo-style human ranking of generated web apps across 18 models including closed-weights models. Claude Sonnet 5 leads at 1333. Inkling scores 1257, tied for seventh overall with Claude Opus 4.6 and competitive with much larger closed-weights models; the lowest shown is 1185. Bar lengths are shown relative to an Elo baseline of 1150."> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Claude Sonnet 5</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 91.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1333</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Claude Fable 5</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 89.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1329</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Claude Opus 4.8</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 67.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1285</span> </div> </div> </div> <div class="interaction-benchmark-bars-row interaction-benchmark-bars-row--open-weight"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">GLM 5.2</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 62.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1275</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Grok 4.5</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 60.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1271</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">GPT-5.6 Sol</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 55%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1260</span> </div> </div> </div> <div class="interaction-benchmark-bars-row interaction-benchmark-bars-row--highlight interaction-benchmark-bars-row--open-weight"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Inkling</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-tml" style="width: 53.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-tml">1257</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Claude Opus 4.6</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 53.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1257</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Gemini 3.5 Flash</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 52%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1254</span> </div> </div> </div> <div class="interaction-benchmark-bars-row interaction-benchmark-bars-row--open-weight"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Kimi K2.6</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 49.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1249</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Claude Sonnet 4.6</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 43.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1237</span> </div> </div> </div> <div class="interaction-benchmark-bars-row interaction-benchmark-bars-row--open-weight"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Kimi K2.7 Code</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 42%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1234</span> </div> </div> </div> <div class="interaction-benchmark-bars-row interaction-benchmark-bars-row--open-weight"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">GLM 5.1</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 41.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1233</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Claude Opus 4.5</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 31%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1212</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Grok 4.20 Reasoning</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 26.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1203</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Gemini 3.1 Pro Preview</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 18.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1187</span> </div> </div> </div> <div class="interaction-benchmark-bars-row"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Grok 4.3</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 17.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1185</span> </div> </div> </div> <div class="interaction-benchmark-bars-row interaction-benchmark-bars-row--open-weight"> <div class="interaction-benchmark-bars-label"> <p class="interaction-benchmark-bars-capability">Kimi K2.5 (Thinking)</p> </div> <div class="interaction-benchmark-bars-series"> <div class="interaction-benchmark-bars-series-row"> <span class="interaction-benchmark-bars-track" aria-hidden="true"><span class="interaction-benchmark-bars-fill interaction-benchmark-bars-fill-baseline" style="width: 17.5%;"></span></span> <span class="interaction-benchmark-bars-value interaction-benchmark-bars-value-baseline">1185</span> </div> </div> </div> </div> </div> <figcaption>Inkling’s position on Design Arena’s Agentic Web Dev leaderboard, a blinded human evaluation of generated apps. Dots refer to open-weights models.</figcaption></figure> <h4 id="cohesively-styled-artifacts">Cohesively styled artifacts</h4> <p>Inkling creates multi-page artifacts with precise instruction following, accurate information, and cohesive styling and design throughout.</p> <figure><div class="demo" data-demo="" id="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt" style="--demo-height: 640px"><div class="demo-header"><div class="demo-tabs" role="tablist"><button class="demo-tab" type="button" role="tab" data-demo-tab="" id="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-tab-0" aria-controls="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-panel-0" aria-selected="true">Result</button><button class="demo-tab" type="button" role="tab" data-demo-tab="" id="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-tab-1" aria-controls="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-panel-1" aria-selected="false" tabindex="-1">Prompt</button></div></div><div class="demo-body"><div class="demo-panel demo-panel-pdf" id="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-panel-0" role="tabpanel" aria-labelledby="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-tab-0" tabindex="0"><iframe data-src="/news/introducing-inkling/files/breakfast-around-the-world.1bc735e9e07c7d196460dcd48ac8b31c.pdf#navpanes=0&amp;zoom=45" title="Result: PDF document" loading="lazy" referrerpolicy="no-referrer"></iframe><a class="demo-pdf-open" href="https://thinkingmachines.ai/news/introducing-inkling/files/breakfast-around-the-world.1bc735e9e07c7d196460dcd48ac8b31c.pdf" target="_blank" rel="noopener">Open PDF ↗</a><div class="demo-pdf-fallback"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"></path><path d="M14 2v4a2 2 0 0 0 2 2h4"></path><path d="M10 9H8"></path><path d="M16 13H8"></path><path d="M16 17H8"></path></svg><p>Your browser downloads PDFs instead of displaying them inline.</p><a href="https://thinkingmachines.ai/news/introducing-inkling/files/breakfast-around-the-world.1bc735e9e07c7d196460dcd48ac8b31c.pdf" target="_blank" rel="noopener">Open the PDF</a></div><noscript><p class="demo-run-fallback"><a href="/news/introducing-inkling/files/breakfast-around-the-world.1bc735e9e07c7d196460dcd48ac8b31c.pdf">Open the PDF</a> (embedded viewer requires JavaScript).</p></noscript></div><div class="demo-panel demo-panel-text" id="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-panel-1" role="tabpanel" aria-labelledby="demo-files-breakfast-around-the-world-pdf-prompts-pdf-txt-tab-1" tabindex="0" hidden=""><pre><code data-lang="text">Create a premium, editorial-style food and travel journal titled: "Breakfast Around the World" Six Mornings, Six Cities Explore how people begin the day in Paris, Tokyo, Istanbul, Mexico City, Hong Kong, and Copenhagen through food, cafés, tableware, local rituals, and the atmosphere of the city in the early morning. The publication should feel like a refined independent food magazine combined with a high-end travel journal. Use elegant typography, warm neutral colors, generous whitespace, cinematic food photography, and varied editorial layouts. Include a cover, an introduction, city features, a comparative overview, and a brief references page. Keep the writing concise, atmospheric, and factually accurate. Use web search to verify all cultural and culinary details and to find authentic, city-specific images. Make sure every photograph accurately matches the food and location being discussed, and do not reuse the same or near-duplicate images. Create a polished PDF of approximately 8–10 pages.</code></pre></div></div></div><figcaption>From the prompt in the second tab, Inkling produces a polished nine-page PDF food and travel journal — browse the actual document above.</figcaption></figure> <h4 id="multiplayer-game-created-through-long-refinement-loop">Multiplayer game created through long refinement loop</h4> <p>Inkling refined an online snake game through 40 iterations of feedback from GPT Codex serving as a reviewer. The ability to sustain a long process of refinement and improve from feedback is crucial to creating the best collaborative work.</p> <figure><div class="demo" data-demo="" id="demo-files-snake-game-mp4-prompts-snake-txt" style="--demo-height: 480px"><div class="demo-header"><div class="demo-tabs" role="tablist"><button class="demo-tab" type="button" role="tab" data-demo-tab="" id="demo-files-snake-game-mp4-prompts-snake-txt-tab-0" aria-controls="demo-files-snake-game-mp4-prompts-snake-txt-panel-0" aria-selected="true">Result</button><button class="demo-tab" type="button" role="tab" data-demo-tab="" id="demo-files-snake-game-mp4-prompts-snake-txt-tab-1" aria-controls="demo-files-snake-game-mp4-prompts-snake-txt-panel-1" aria-selected="false" tabindex="-1">Prompt</button></div></div><div class="demo-body"><div class="demo-panel demo-panel-video" id="demo-files-snake-game-mp4-prompts-snake-txt-panel-0" role="tabpanel" aria-labelledby="demo-files-snake-game-mp4-prompts-snake-txt-tab-0" tabindex="0"><video controls="" playsinline="" preload="none" src="https://thinkingmachines.ai/news/introducing-inkling/files/snake-game.31495dc01b5f14bf80013af4391098db.mp4" poster="https://thinkingmachines.ai/news/introducing-inkling/images/snake-game-poster.d1880cb231040fdcb3acc7abc45c6e04.jpg" width="1280" height="802" aria-label="Screen recording of a generated multiplayer snake game: players and bots share a circular arena with food pellets, boost trails, and a live leaderboard"></video></div><div class="demo-panel demo-panel-text" id="demo-files-snake-game-mp4-prompts-snake-txt-panel-1" role="tabpanel" aria-labelledby="demo-files-snake-game-mp4-prompts-snake-txt-tab-1" tabindex="0" hidden=""><pre><code data-lang="text">Build a multiplayer snake game: a server-authoritative real-time simulation where players and bots share one circular arena, played in the browser. TypeScript on both server and client (Node.js + `ws`), plain HTML5 Canvas client. ## Environment and verification constraints (read first) - Toolchain: Node.js v24, npm 11. Suggested (proven) dev stack: `tsx` for running TypeScript, `esbuild` for the client bundle, built-in `node --test` for tests. Runtime dependency: `ws` only. ## Requirements 1. Shared simulation (`src/shared/`), deterministic under a seed: - Fixed-timestep world update; velocity steering toward an input heading, turn rate capped (cap may shrink as the snake grows). - Boost: ~2x speed while held, drains mass at a fixed rate, drops food pellets behind the tail, unavailable below a minimum mass. - Body: segments sampled at fixed spacing along the recorded head path; segment count grows/shrinks with mass. - Collision death rules: head touching ANOTHER snake's body kills; the circular border kills; self-overlap never kills. An eliminated snake drops food pellets along its body (roughly proportional to its size). Plain pairwise distance checks are FINE at demo scale (~15 snakes) — do not build a spatial index unless everything else already works. - Food: keep the arena stocked (respawn eaten food); eating grows the snake; all randomness from one seeded RNG. - Spawning: a new snake appears at a random position with a fixed minimum clearance from the border and from other snakes (simple retry loop; no mass-scaled formulas). - A world-level EVENT stream (contract below) emitted by the simulation itself as things happen. 2. Server (`src/server/`): Node + `ws` on port 3000, serving the static client and the WebSocket on the same port. Join (nickname) / input ({heading, boost}) / full-world snapshot broadcast. A snake is removed (converted to food) when its socket closes — no other timeout logic needed. Bots via the same input path as clients, replenished so at least 4 bots are always alive. Top-10 leaderboard by length in every snapshot. 3. Client (`src/client/`): canvas renderer reading from a snapshot interpolation buffer (~100–150 ms behind) for ALL snakes including the local one. Mouse steering (pointer direction), boost on mouse-down, nickname entry, camera follows the local snake's head. Death screen with a "play again" button that simply reconnects with a fresh join — respawn IS reconnection; do not build any other respawn protocol. 4. Headless simulate harness (`src/simulate.ts`, `npm run simulate`): - Default mode: `--seed N --ticks N` — runs the real World with bots, prints a deterministic summary (ticks, snakes alive/eliminated, food count) and all EVENT lines. - Scenario mode: `--scenario &lt;name&gt; --seed N` — runs the real World with scripted inputs. Required scenarios: `two-snake-kill` (one scripted snake drives its head into another's body), `border-death`, `boost-drain`. 5. Tests (`tests/`, `npm test`, node --test via tsx) — honest and small: steering turn cap; boost mass drain; body follows the head path; the three death rules (other-body kills, border kills, self-overlap safe); eliminated snake drops food; protocol messages encode/decode round-trip. Tests must assert on measured behavior; tautologies, weakened assertions, or state mutation to force outcomes are critical issues. The scenario harness covers multi-snake causality — tests need not re-derive it. 6. `README.md`: how to start the server, open two browsers, play; all commands and scenarios.</code></pre></div></div></div><figcaption>A multiplayer snake game generated by Inkling from the prompt in the second tab — real-time server, bots, leaderboard and all.</figcaption></figure> <h3 id="controllable-thinking-effort">Controllable thinking effort</h3> <p>Test-time scaling and problem-solving are the core capability of every model, but that capacity is hard to capture with a single number. Developers fine-tuning models for a specialized task <a href="https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/">care as much about efficiency</a> as about the max-effort performance on a public benchmark. Cost and latency are often binding constraints in real-world applications, and low latency in particular is crucial for enabling collaboration and improvement through iteration.</p> <figure> <div class="effort-curve-figure intelligence-interactivity-figure" data-plot-point-stagger="18" data-plot-point-jitter="10"> <div class="effort-curve"> <div class="effort-legend intelligence-interactivity-legend" aria-label="Model legend"> <span class="ii-legend-item"><span class="effort-sweep-marker" aria-hidden="true"></span>Inkling (effort sweep)</span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--e-glm);"></span>GLM-5.2</span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--e-kimi26);"></span>Kimi K2.6</span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--e-nemotron);"></span>Nemotron 3 Ultra</span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--e-kimi25);"></span>Kimi K2.5</span> <span class="ii-legend-item"><span class="ii-legend-marker ii-marker-circle" style="--ii-marker-color: var(--e-gptoss);"></span>GPT-OSS (high)</span> </div> <div class="benchmark-results-controls chart-hscroll-controls" role="group" aria-label="Effort curve chart controls"> <button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll effort curve chart left" title="Scroll left"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg> </button> <button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll effort curve chart right" title="Scroll right"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg> </button> </div> <div class="effort-scroll intelligence-interactivity-charts chart-hscroll"> <div class="effort-chart intelligence-interactivity-chart" role="img" aria-label="Three panels — Terminal Bench 2.1, Humanity's Last Exam, and IFBench — plot score against mean generated tokens, each on its own linear axis (0 to 100k, 50k, and 10k tokens respectively). Inkling's controllable-effort sweep traces a curve; competing models are shown as single operating points. Across all three, Inkling reaches a given score at fewer tokens."> <svg class="effort-svg intelligence-interactivity-svg" viewBox="0 40 924 412" aria-hidden="true" focusable="false"> <g class="effort-panel"> <text class="effort-title ii-panel-title" x="177.0" y="58">Terminal Bench 2.1</text> <g class="ii-grid"> <line x1="66" y1="360.0" x2="288" y2="360.0"></line> <line x1="66" y1="268.0" x2="288" y2="268.0"></line> <line x1="66" y1="176.0" x2="288" y2="176.0"></line> <line x1="66" y1="84.0" x2="288" y2="84.0"></line> <line x1="66.0" y1="84" x2="66.0" y2="360"></line> <line x1="121.5" y1="84" x2="121.5" y2="360"></line> <line x1="177.0" y1="84" x2="177.0" y2="360"></line> <line x1="232.5" y1="84" x2="232.5" y2="360"></line> <line x1="288.0" y1="84" x2="288.0" y2="360"></line> </g> <g class="ii-axis"> <line x1="66" y1="84" x2="66" y2="360"></line> <line x1="66" y1="360" x2="288" y2="360"></line> </g> <g class="ii-ticks ii-y-ticks"> <text x="56" y="366.0">20%</text> <text x="56" y="274.0">40%</text> <text x="56" y="182.0">60%</text> <text x="56" y="90.0">80%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="66.0" y="392">0</text> <text x="121.5" y="392">25k</text> <text x="177.0" y="392">50k</text> <text x="232.5" y="392">75k</text> <text x="288.0" y="392">100k</text> </g> <polyline class="effort-line effort-hero effort-hero-line effort-hero-series" style="--effort-series-delay: 0ms;" points="75.7,287.3 81.2,245.4 92.5,192.6 108.4,177.9 142.8,170.6 173.2,172.3"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(248.3 93.7)"><circle class="ii-point-marker effort-glm" r="5"></circle></g> <g class="ii-point" transform="translate(159.8 129.1)"><circle class="ii-point-marker effort-kimi26" r="5"></circle></g> <g class="ii-point" transform="translate(138.8 191.7)"><circle class="ii-point-marker effort-nemotron" r="5"></circle></g> <g class="ii-point" transform="translate(107.5 224.3)"><circle class="ii-point-marker effort-kimi25" r="5"></circle></g> <g class="ii-point" transform="translate(75.7 287.3)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(81.2 245.4)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(92.5 192.6)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(108.4 177.9)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(142.8 170.6)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(173.2 172.3)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> </g> </g> <g class="effort-panel"> <text class="effort-title ii-panel-title" x="465.0" y="58">Humanity's Last Exam*</text> <g class="ii-grid"> <line x1="354" y1="360.0" x2="576" y2="360.0"></line> <line x1="354" y1="268.0" x2="576" y2="268.0"></line> <line x1="354" y1="176.0" x2="576" y2="176.0"></line> <line x1="354" y1="84.0" x2="576" y2="84.0"></line> <line x1="354.0" y1="84" x2="354.0" y2="360"></line> <line x1="409.5" y1="84" x2="409.5" y2="360"></line> <line x1="465.0" y1="84" x2="465.0" y2="360"></line> <line x1="520.5" y1="84" x2="520.5" y2="360"></line> <line x1="576.0" y1="84" x2="576.0" y2="360"></line> </g> <g class="ii-axis"> <line x1="354" y1="84" x2="354" y2="360"></line> <line x1="354" y1="360" x2="576" y2="360"></line> </g> <g class="ii-ticks ii-y-ticks"> <text x="344" y="366.0">10%</text> <text x="344" y="274.0">20%</text> <text x="344" y="182.0">30%</text> <text x="344" y="90.0">40%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="354.0" y="392">0</text> <text x="409.5" y="392">12.5k</text> <text x="465.0" y="392">25k</text> <text x="520.5" y="392">37.5k</text> <text x="576.0" y="392">50k</text> </g> <polyline class="effort-line effort-hero effort-hero-line effort-hero-series" style="--effort-series-delay: 65ms;" points="359.0,360.0 362.4,351.7 368.8,319.5 395.4,245.9 430.1,218.3 464.3,198.1 503.2,176.0"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(536.3 84.0)"><circle class="ii-point-marker effort-glm" r="5"></circle></g> <g class="ii-point" transform="translate(545.1 121.8)"><circle class="ii-point-marker effort-kimi26" r="5"></circle></g> <g class="ii-point" transform="translate(518.4 203.6)"><circle class="ii-point-marker effort-nemotron" r="5"></circle></g> <g class="ii-point" transform="translate(466.9 181.5)"><circle class="ii-point-marker effort-kimi25" r="5"></circle></g> <g class="ii-point" transform="translate(451.7 281.8)"><circle class="ii-point-marker effort-gptoss" r="5"></circle></g> <g class="ii-point" transform="translate(359.0 360.0)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(362.4 351.7)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(368.8 319.5)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(395.4 245.9)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(430.1 218.3)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(464.3 198.1)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(503.2 176.0)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> </g> </g> <g class="effort-panel"> <text class="effort-title ii-panel-title" x="753.0" y="58">IFBench</text> <g class="ii-grid"> <line x1="642" y1="360.0" x2="864" y2="360.0"></line> <line x1="642" y1="268.0" x2="864" y2="268.0"></line> <line x1="642" y1="176.0" x2="864" y2="176.0"></line> <line x1="642" y1="84.0" x2="864" y2="84.0"></line> <line x1="642.0" y1="84" x2="642.0" y2="360"></line> <line x1="686.4" y1="84" x2="686.4" y2="360"></line> <line x1="730.8" y1="84" x2="730.8" y2="360"></line> <line x1="775.2" y1="84" x2="775.2" y2="360"></line> <line x1="819.6" y1="84" x2="819.6" y2="360"></line> <line x1="864.0" y1="84" x2="864.0" y2="360"></line> </g> <g class="ii-axis"> <line x1="642" y1="84" x2="642" y2="360"></line> <line x1="642" y1="360" x2="864" y2="360"></line> </g> <g class="ii-ticks ii-y-ticks"> <text x="632" y="366.0">60%</text> <text x="632" y="274.0">70%</text> <text x="632" y="182.0">80%</text> <text x="632" y="90.0">90%</text> </g> <g class="ii-ticks ii-x-ticks"> <text x="642.0" y="392">0</text> <text x="686.4" y="392">2k</text> <text x="730.8" y="392">4k</text> <text x="775.2" y="392">6k</text> <text x="819.6" y="392">8k</text> <text x="864.0" y="392">10k</text> </g> <polyline class="effort-line effort-hero effort-hero-line effort-hero-series" style="--effort-series-delay: 130ms;" points="658.9,268.0 669.4,222.9 685.8,194.5 716.6,182.5 748.1,172.3 769.1,183.4 782.9,177.0"></polyline> <g class="ii-points"> <g class="ii-point" transform="translate(787.3 237.6)"><circle class="ii-point-marker effort-glm" r="5"></circle></g> <g class="ii-point" transform="translate(830.6 212.8)"><circle class="ii-point-marker effort-kimi26" r="5"></circle></g> <g class="ii-point" transform="translate(752.1 163.1)"><circle class="ii-point-marker effort-nemotron" r="5"></circle></g> <g class="ii-point" transform="translate(774.3 268.0)"><circle class="ii-point-marker effort-kimi25" r="5"></circle></g> <g class="ii-point" transform="translate(793.0 277.2)"><circle class="ii-point-marker effort-gptoss" r="5"></circle></g> <g class="ii-point" transform="translate(658.9 268.0)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(669.4 222.9)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(685.8 194.5)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(716.6 182.5)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(748.1 172.3)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(769.1 183.4)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> <g class="ii-point" transform="translate(782.9 177.0)"><circle class="ii-point-marker effort-hero" r="4"></circle></g> </g> </g> <text class="effort-xlabel ii-axis-label" x="462" y="438">Mean generated tokens</text> </svg> </div> </div> </div> </div> <figcaption>Sweeping Inkling’s effort setting from 0.2 to 0.99 traces its performance against mean generated tokens on Terminal Bench 2.1, HLE, and IFBench; competing models are shown at their default operating point. Inkling reaches a given score at fewer tokens — for example, it matches Nemotron 3 Ultra on Terminal Bench 2.1 at roughly a third of the tokens. *Humanity’s Last Exam scores reflect an earlier checkpoint and run slightly below the final release.</figcaption></figure> <p>Inkling supports controllable thinking effort, allowing you to balance performance with token efficiency. The chart above shows the effort/performance curve of Inkling as well as other open-weights models on a range of benchmarks: Terminal Bench 2.1 for agentic coding, HLE for advanced reasoning, and IFBench for instruction following. Inkling spends one third as many tokens to achieve the same performance as Nemotron 3 Ultra on Terminal Bench. Cost and latency matter for a model that you run millions of times and as part of longer workflows; looking at the full cost curve allows developers to choose the best model for each use case.</p> <h3 id="multimodality">Multimodality</h3> <p>A major goal of Inkling’s design is to serve as the background reasoning model in the <a href="https://thinkingmachines.ai/blog/interaction-models/">interaction models system</a> we recently introduced. Interaction models enable the user to collaborate naturally, using voice and vision in real time. This requires a model natively trained for broad multimodal capabilities.</p> <div class="inkling-eval-table-block" style="--benchmark-model-count: 7"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Multimodal benchmarks table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Multimodal benchmarks table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Multimodal benchmarks table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="7"></colgroup><thead><tr class="benchmark-model-group-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model-group" colspan="5" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Open weights</span></span></th><th class="benchmark-model-group" colspan="2" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Closed weights</span></span></th></tr><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling</span></span><span class="benchmark-subtitle">effort=0.99</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3-Omni</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Nemotron-3</span><span class="benchmark-model-name-line">Nano-Omni</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Kimi K2.5</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Kimi K2.6</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Qwen3.5</span><span class="benchmark-model-name-line">Omni-Plus</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Gemini 3.1 Pro</span><span class="benchmark-model-name-line">(high)</span></span></th></tr></thead><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Audio</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Audio MC</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">56.6%</span></td><td class="benchmark-value">24.3%</td><td class="benchmark-value">23.2%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">37.6%</td><td class="benchmark-value">66.8%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMAU</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.2%</span></td><td class="benchmark-value">77.5%</td><td class="benchmark-value">76.7%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">81.1%</td><td class="benchmark-value">82.5%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">VoiceBench</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">91.4%</span></td><td class="benchmark-value">88.8%</td><td class="benchmark-value">89.4%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">92.4%</td><td class="benchmark-value">94.3%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Vision</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMMU Pro<br>(Standard 10)</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">73.5%</span></td><td class="benchmark-value">60.0%</td><td class="benchmark-value">53.0%</td><td class="benchmark-value">75.0%</td><td class="benchmark-value">79.0%</td><td class="benchmark-value">71.0%</td><td class="benchmark-value">82.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">78.1%</span></td><td class="benchmark-value">61.1%</td><td class="benchmark-value">63.6%</td><td class="benchmark-value">77.5%</td><td class="benchmark-value">80.4%</td><td class="benchmark-value">72.5%</td><td class="benchmark-value">80.2%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span><span class="benchmark-subtitle">with python</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">82.0%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">78.7%</td><td class="benchmark-value">86.7%</td><td class="benchmark-value">–</td><td class="benchmark-value">89.9%</td></tr></tbody></table></div></div> <p class="benchmark-results-note">Audio and vision benchmarks against specialist omni models (open- and closed-weight), reported at effort=0.99.</p> <p>The multimodal components were trained from scratch on general-domain data. We opted for an encoder-free architecture for audio and vision inputs, consistent with the interaction model design. Audio signals are input as dMel spectrograms<label for="sn-dmel" class="sidenote-number"></label><input type="checkbox" id="sn-dmel" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2407.15835">dMel: Speech Tokenization made Simple</a> (Richard He Bai et al, 2024)</span>, while images are encoded as patches of 40x40 pixels using a four-layer hMLP<label for="sn-hmlp" class="sidenote-number"></label><input type="checkbox" id="sn-hmlp" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2203.09795">Three things everyone should know about Vision Transformers</a> (Hugo Touvron et al, 2022)</span>. Both are transformed via a light-weight embedding layer and processed jointly with text tokens.</p> <p>Inkling transcribes speech, follows spoken instructions, answers questions about recordings, and reasons over longer-form audio. These capabilities place it among the strongest open-weights audio models on VoiceBench, MMAU, and AudioMC. For vision, Inkling accepts images as input and can describe visual content, answer questions, and perform in-depth reasoning based on the provided visual information. It demonstrates strong performance on charts, diagrams, and mathematical visual reasoning tasks. During inference, Inkling can also leverage a Python tool to support image understanding through operations such as zooming and cropping, while seamlessly integrating visual reasoning with code-based reasoning.</p> <p>As our first release, Inkling establishes a robust multimodal foundation for future work. We expect its multimodal capabilities to continue improving as we expand the model and training pipeline in subsequent iterations.</p> <h3 id="epistemics">Epistemics</h3> <p>We trained Inkling for calibration, instruction following, and resistance to censorship, which we refer to collectively as the model’s <em>epistemics</em>.</p> <p>Getting the facts right requires more than memorizing a large corpus of knowledge. A useful model must be well-calibrated, expressing the right amount of confidence in its answers — including on questions which aren’t yet settled. The latter is a crucial capability for prediction and forecasting, an important use case where <a href="https://thinkingmachines.ai/news/training-llms-to-predict-world-events/">fine-tuned models have shown rapid improvement</a> in recent months, <a href="https://x.com/tinkerapi/status/2056798250532057139">outperforming frontier LLMs</a>.</p> <div class="benchmark-results-controls" role="group" aria-label="Forecasting results table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll forecasting table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll forecasting table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div> <div class="benchmark-results-scroll benchmark-results-scroll--six-model"><table class="benchmark-results-table"><thead><tr><td class="benchmark-heading-spacer" colspan="1" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content">Inkling</span></th><th class="benchmark-model">GPT-5.5</th><th class="benchmark-model">Claude Opus 4.8</th><th class="benchmark-model">Gemini 3.1 Pro</th><th class="benchmark-model">Grok 4.3</th><th class="benchmark-model">Kimi K2.6</th></tr></thead><tbody><tr><td class="benchmark-name"><span class="benchmark-title">ForecastBench</span><span class="benchmark-subtitle">Brier Index ↑ · no search</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">61.1 ± 0.79</span></td><td class="benchmark-value">59.1 ± 0.41</td><td class="benchmark-value">54.6 ± 0.99</td><td class="benchmark-value">61.1 ± 0.56</td><td class="benchmark-value">61.7 ± 0.54</td><td class="benchmark-value">58.1 ± 0.54</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">ForecastBench</span><span class="benchmark-subtitle">Brier Index ↑ · with search</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">63.7 ± 0.82</span></td><td class="benchmark-value">64.7 ± 1.28</td><td class="benchmark-value">58.6 ± 0.66</td><td class="benchmark-value">64.3 ± 1.05</td><td class="benchmark-value">63.2 ± 0.82</td><td class="benchmark-value">–</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Prophet Arena</span><span class="benchmark-subtitle">Brier Score ↓</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">0.1617</span></td><td class="benchmark-value">0.1598</td><td class="benchmark-value">0.1605</td><td class="benchmark-value">0.1594</td><td class="benchmark-value">0.1715</td><td class="benchmark-value">0.1675</td></tr></tbody></table></div> <p class="benchmark-results-note">Results were obtained during testing between June 30 and July 13, 2026 on a different checkpoint of Inkling than the one released.</p> <p>Forecasting requires integrating multiple sources of information into a calibrated probability, a core skill for a model users can trust. A model that’s confident in every answer it gives, including when it’s missing info and confabulates, forces the user to double-check everything. A model that gives the appropriate measure of confidence is useful across more real-world domains where information is often conflicting, unreliable, or hard to find. We trained for calibration with RL against proper scoring rules on a large corpus of resolved real-world questions.</p> <p>The second component of a trustworthy model is instruction following, including on hard-to-verify, complex queries. We did RL with two automated graders: a rubric grader and claims grader. The first grader scores each response against a checklist of what a good answer should contain. Rubrics can penalize errors in principle, but in practice they emphasize recall and can be hacked by models spraying plausibly relevant facts hoping to match rubric items. The claims grader verifies each factual claim in the response, penalizing claims that don’t check out. It performs agentic web search for claim verification, not relying solely on its own knowledge. Together, the two graders improve helpfulness and reduce hallucination at the same time, rather than trading one for the other.</p> <p>These rewards don’t directly target calibrated uncertainty in long-form responses, so we added targeted datasets that do. The largest is short-form factual QA with abstention-aware rewards: answering only pays off when the model is likely to be right, so the optimal policy is to answer when confident and otherwise say “I don’t know” or give a hedged best guess. Some prompts encourage or forbid hedging, teaching the model to follow the user’s preference for a forced guess versus a calibrated non-answer.</p> <p>Finally, we trained Inkling to answer directly on topics that may be subject to censorship. Cognition evaluated the model on their Propaganda and Censorship Eval<label for="sn-censorship" class="sidenote-number"></label><input type="checkbox" id="sn-censorship" class="margin-toggle"><span class="sidenote">The Cognition Team, “<a href="https://cognition.com/blog/measuring-open-source-model-trustworthiness">Measuring the Trustworthiness of Open-Source-Derived Models</a>,” 2026.</span>, and it exhibited strong patterns of censorship non-compliance.</p> <h3 id="safety">Safety</h3> <p>We trained Inkling to an internal spec of safe model behavior across all modalities. We then commissioned external safety testers to verify the results.</p> <p>We evaluated Inkling’s safety in several areas. For dangerous capabilities — CBRN, cyber, and loss of control — we ran internal evaluations and enlisted external testers. We attended to human-AI threat vectors, including sycophancy, vulnerable users, and harmful manipulation, using internal evaluations and external testers.</p> <div class="benchmark-results-controls" role="group" aria-label="Safety results table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll safety table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll safety table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div> <div class="benchmark-results-scroll benchmark-results-scroll--six-model"><table class="benchmark-results-table"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="6"></colgroup><thead><tr><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content">Inkling<span class="benchmark-subtitle">effort=0.99</span></span></th><th class="benchmark-model">Nemotron 3<br>Ultra</th><th class="benchmark-model">Kimi K2.5</th><th class="benchmark-model">Kimi K2.6</th><th class="benchmark-model">GLM 5.2</th><th class="benchmark-model">DeepSeek V4<br>Pro</th></tr></thead><tbody><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS (Adversarial)</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">78.0%</span></td><td class="benchmark-value">77.6%</td><td class="benchmark-value">54.1%</td><td class="benchmark-value">65.6%</td><td class="benchmark-value">71.3%</td><td class="benchmark-value">36.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS (Benign)</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">95.9%</span></td><td class="benchmark-value">90.5%</td><td class="benchmark-value">98.3%</td><td class="benchmark-value">97.2%</td><td class="benchmark-value">90.0%</td><td class="benchmark-value">98.5%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">StrongREJECT</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">98.6%</span></td><td class="benchmark-value">98.7%</td><td class="benchmark-value">99.5%</td><td class="benchmark-value">99.8%</td><td class="benchmark-value">98.5%</td><td class="benchmark-value">98.6%</td></tr></tbody></table></div> <p>Inkling shows the strongest built-in safeguards of any open-weights model we compared on FORTRESS, a benchmark that tests refusal of requests related to weapons and violence alongside benign look-alike queries. Inkling refused more harmful requests without over-refusing benign analogs. Inkling scores above 98% on StrongREJECT — a refusal test of unambiguous harmful requests — in line with other open and closed-weights models.</p> <p>Safety is crucial for open-weights models. We’re continuing to study safety behavior and capability uplift in customizable models, including how safety behavior is impacted by fine-tuning on Tinker.</p> <h2 id="benchmarking-inkling">Benchmarking Inkling</h2> <p>We benchmark Inkling on a broad range of capabilities. All evals are run at effort 0.99 and temperature 1.0. All coding evals run with 256K max-token trajectory limit.</p> <p>To improve consistency, we rely on externally reported evaluations for both internal and external models when applicable. Specifically, we use the score reported by Artificial Analysis for the following evals: Humanity’s Last Exam, GPQA Diamond, GDPVal, Tau 3 Banking, AA Omniscience, MMMU Pro.</p> <div class="inkling-eval-table-block" style="--benchmark-model-count: 9"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Inkling evaluation table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Inkling evaluation table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Inkling evaluation table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="9"></colgroup><thead><tr class="benchmark-model-group-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model-group" colspan="6" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Open weights</span></span></th><th class="benchmark-model-group" colspan="3" scope="colgroup"><span class="benchmark-model-group-anchor"><span class="benchmark-model-group-label">Closed weights</span></span></th></tr><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model benchmark-instant-start"><span class="benchmark-lock-content"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling</span></span><span class="benchmark-subtitle">effort=0.99</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Nemotron 3</span><span class="benchmark-model-name-line">Ultra</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Kimi K2.5</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Kimi K2.6</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">GLM 5.2</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">DeepSeek V4</span><span class="benchmark-model-name-line">Pro</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Gemini 3.1 Pro</span><span class="benchmark-model-name-line">(high)</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Claude Fable 5</span><span class="benchmark-model-name-line">(max)</span></span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">GPT 5.6 Sol</span><span class="benchmark-model-name-line">(max/xhigh)</span></span></th></tr></thead><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Reasoning</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HLE</span><span class="benchmark-subtitle">text only</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">29.7%</span></td><td class="benchmark-value">26.6%</td><td class="benchmark-value">29.4%</td><td class="benchmark-value">35.9%</td><td class="benchmark-value">40.1%</td><td class="benchmark-value">35.9%</td><td class="benchmark-value">44.7%</td><td class="benchmark-value">53.3%</td><td class="benchmark-value">47.2%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HLE</span><span class="benchmark-subtitle">with tools</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">46.0%</span></td><td class="benchmark-value">37.4%</td><td class="benchmark-value">50.2%</td><td class="benchmark-value">54.0%</td><td class="benchmark-value">54.7%</td><td class="benchmark-value">48.2%</td><td class="benchmark-value">51.4%</td><td class="benchmark-value">64.5%</td><td class="benchmark-value">55.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AIME 2026</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">97.1%</span></td><td class="benchmark-value">94.2%</td><td class="benchmark-value">95.8%</td><td class="benchmark-value">96.4%</td><td class="benchmark-value">99.2%</td><td class="benchmark-value">96.7%</td><td class="benchmark-value">98.3%</td><td class="benchmark-value">99.9%</td><td class="benchmark-value">99.9%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">GPQA Diamond</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">87.2%</span></td><td class="benchmark-value">86.7%</td><td class="benchmark-value">87.9%</td><td class="benchmark-value">91.1%</td><td class="benchmark-value">89.5%</td><td class="benchmark-value">88.8%</td><td class="benchmark-value">94.1%</td><td class="benchmark-value">92.6%</td><td class="benchmark-value">94.1%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Agentic (coding)</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SWEBench Verified<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/introducing-inkling/#eval-note-swebench-verified" aria-label="Read the SWEBench Verified evaluation note"><sup>*</sup></a></span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.6%</span></td><td class="benchmark-value">70.7%</td><td class="benchmark-value">76.8%</td><td class="benchmark-value">80.2%</td><td class="benchmark-value">80.0%</td><td class="benchmark-value">80.6%</td><td class="benchmark-value">80.6%</td><td class="benchmark-value">95.0%</td><td class="benchmark-value">82.2%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SWEBench Pro</span><span class="benchmark-subtitle">Public</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">54.3%</span></td><td class="benchmark-value">46.4%</td><td class="benchmark-value">50.7%</td><td class="benchmark-value">58.6%</td><td class="benchmark-value">62.1%</td><td class="benchmark-value">55.4%</td><td class="benchmark-value">54.2%</td><td class="benchmark-value">80.0%</td><td class="benchmark-value">64.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Terminal Bench 2.1<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/introducing-inkling/#eval-note-terminal-bench-2-1" aria-label="Read the Terminal Bench 2.1 evaluation note"><sup>*</sup></a></span><span class="benchmark-subtitle">Best Harness</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">63.8%</span></td><td class="benchmark-value">56.4%</td><td class="benchmark-value">51.3%</td><td class="benchmark-value">71.3%</td><td class="benchmark-value">82.7%</td><td class="benchmark-value">64.0%</td><td class="benchmark-value">73.8%</td><td class="benchmark-value">84.6%</td><td class="benchmark-value">89.5%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Agentic (general)</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">GDPVal-AA v2</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">1238</span></td><td class="benchmark-value">1164</td><td class="benchmark-value">1009</td><td class="benchmark-value">1190</td><td class="benchmark-value">1514</td><td class="benchmark-value">1307</td><td class="benchmark-value">962</td><td class="benchmark-value">1760</td><td class="benchmark-value">1748</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MCP Atlas</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">76.0%</span></td><td class="benchmark-value">44.7%</td><td class="benchmark-value">64.0%</td><td class="benchmark-value">68.1%</td><td class="benchmark-value">77.8%</td><td class="benchmark-value">73.2%</td><td class="benchmark-value">78.2%</td><td class="benchmark-value">83.3%</td><td class="benchmark-value">81.8%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Tau 3 Banking</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">23.7%</span></td><td class="benchmark-value">13.8%</td><td class="benchmark-value">14.2%</td><td class="benchmark-value">20.6%</td><td class="benchmark-value">26.8%</td><td class="benchmark-value">25.8%</td><td class="benchmark-value">16.5%</td><td class="benchmark-value">26.8%</td><td class="benchmark-value">33.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Toolathlon Verified</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">45.5%</span></td><td class="benchmark-value">34.3%</td><td class="benchmark-value">33.0%</td><td class="benchmark-value">58.0%</td><td class="benchmark-value">59.9%</td><td class="benchmark-value">55.9%</td><td class="benchmark-value">61.1%</td><td class="benchmark-value">76.4%</td><td class="benchmark-value">73.1%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">BrowseComp</span><span class="benchmark-subtitle">w/ ctx management</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.1%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">74.9%</td><td class="benchmark-value">83.2%</td><td class="benchmark-value">–</td><td class="benchmark-value">83.4%</td><td class="benchmark-value">85.9%</td><td class="benchmark-value">88.0%</td><td class="benchmark-value">90.8%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Factuality</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SimpleQA Verified</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">43.9%</span></td><td class="benchmark-value">32.4%</td><td class="benchmark-value">36.9%</td><td class="benchmark-value">38.7%</td><td class="benchmark-value">38.1%</td><td class="benchmark-value">57.0%</td><td class="benchmark-value">77.3%</td><td class="benchmark-value">68.3%</td><td class="benchmark-value">71.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AA Omniscience</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">2.1</span></td><td class="benchmark-value">-1.0</td><td class="benchmark-value">-8.0</td><td class="benchmark-value">6.0</td><td class="benchmark-value">4.0</td><td class="benchmark-value">-10.0</td><td class="benchmark-value">33.0</td><td class="benchmark-value">40.0</td><td class="benchmark-value">22.0</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Chat</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">IFBench</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">79.8%</span></td><td class="benchmark-value">81.4%</td><td class="benchmark-value">70.2%</td><td class="benchmark-value">76.0%</td><td class="benchmark-value">73.3%</td><td class="benchmark-value">76.5%</td><td class="benchmark-value">77.1%</td><td class="benchmark-value">63.5%</td><td class="benchmark-value">72.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Global-MMLU-Lite</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">88.7%</span></td><td class="benchmark-value">85.6%</td><td class="benchmark-value">84.0%</td><td class="benchmark-value">88.4%</td><td class="benchmark-value">89.2%</td><td class="benchmark-value">89.3%</td><td class="benchmark-value">92.7%</td><td class="benchmark-value">93.3%</td><td class="benchmark-value">91.8%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Vision</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMMU Pro</span><span class="benchmark-subtitle">Standard 10</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">73.5%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">75.0%</td><td class="benchmark-value">79.0%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">82.0%</td><td class="benchmark-value">84.2%</td><td class="benchmark-value">83.0%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">78.1%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">77.5%</td><td class="benchmark-value">80.4%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">80.2%</td><td class="benchmark-value">86.5%</td><td class="benchmark-value">84.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/introducing-inkling/#eval-note-charxiv-rq-tools" aria-label="Read the Charxiv RQ evaluation note"><sup>†</sup></a></span><span class="benchmark-subtitle">with python</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">82.0%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">78.7%</td><td class="benchmark-value">86.7%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">89.9%</td><td class="benchmark-value">89.4%</td><td class="benchmark-value">87.8%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Audio</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Audio MC<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/introducing-inkling/#eval-note-audio-mc" aria-label="Read the Audio MC evaluation note"><sup>†</sup></a></span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">56.6%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">66.8%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMAU</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">77.2%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">82.5%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">VoiceBench<a class="benchmark-footnote-ref" href="https://thinkingmachines.ai/news/introducing-inkling/#eval-note-voicebench" aria-label="Read the VoiceBench evaluation note"><sup>†</sup></a></span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">91.4%</span></td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td><td class="benchmark-value">94.3%</td><td class="benchmark-value">–</td><td class="benchmark-value">–</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Safety</span></th><td class="benchmark-category-cell benchmark-value benchmark-instant-start" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">Adversarial</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">78.0%</span></td><td class="benchmark-value">77.6%</td><td class="benchmark-value">54.1%</td><td class="benchmark-value">65.6%</td><td class="benchmark-value">71.3%</td><td class="benchmark-value">36.0%</td><td class="benchmark-value">65.2%</td><td class="benchmark-value">96.0%</td><td class="benchmark-value">82.4%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">Benign</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">95.9%</span></td><td class="benchmark-value">90.5%</td><td class="benchmark-value">98.3%</td><td class="benchmark-value">97.2%</td><td class="benchmark-value">90.0%</td><td class="benchmark-value">98.5%</td><td class="benchmark-value">98.0%</td><td class="benchmark-value">55.1%</td><td class="benchmark-value">98.1%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">StrongREJECT</span></td><td class="benchmark-value benchmark-instant-start"><span class="benchmark-lock-content">98.6%</span></td><td class="benchmark-value">98.7%</td><td class="benchmark-value">99.5%</td><td class="benchmark-value">99.8%</td><td class="benchmark-value">98.5%</td><td class="benchmark-value">98.6%</td><td class="benchmark-value">98.0%</td><td class="benchmark-value">98.7%</td><td class="benchmark-value">98.5%</td></tr></tbody></table></div></div> <p class="benchmark-results-note"><span class="benchmark-results-note-item" id="eval-note-swebench-verified"><span class="benchmark-results-note-label">*SWEBench Verified:</span> Inkling numbers are reported using a bash-only harness. We use self-reported numbers for external models.</span><span class="benchmark-results-note-item" id="eval-note-terminal-bench-2-1"><span class="benchmark-results-note-label">*Terminal Bench 2.1:</span> Inkling numbers are reported using an internal coding harness. A small number of solutions were found to be contaminated from web search and were assigned a score of 0. We use self-reported numbers for external models where available. Otherwise, we report performance using our internal harness.</span><span class="benchmark-results-note-item" id="eval-note-audio-mc"><span class="benchmark-results-note-label">†Audio MC:</span> Other models were evaluated internally since they are not on the official leaderboard.</span><span class="benchmark-results-note-item" id="eval-note-voicebench"><span class="benchmark-results-note-label">†VoiceBench:</span> VoiceBench uses rule-based, hard-coded string matching for grading, making the evaluation sensitive to output-formatting differences. We therefore added a system message instructing models to follow the expected answer format.</span><span class="benchmark-results-note-item" id="eval-note-charxiv-rq-tools"><span class="benchmark-results-note-label">†CharXiv RQ with tools:</span> We benchmarked Claude Fable 5 and GPT 5.6 Sol (max/xhigh) using our internal Python harness.</span></p> <h2 id="the-making-of-inkling">The making of Inkling</h2> <h3 id="architecture">Architecture</h3> <p>Inkling is a Mixture-of-Experts Transformer with a handful of departures from the common recipe, each chosen for efficiency and long-context performance.</p> <p>The MoE design largely follows DeepSeek-V3. Each MoE layer contains 256 routed experts and 2 shared experts, with 6 routed experts active per token. Inkling uses a sigmoid-based router with an auxiliary-loss-free load-balancing bias. The scores of the selected routed experts and the shared experts are normalized jointly and used to weight their combined outputs.</p> <p>For attention, we interleave sliding-window and global layers at a 5:1 ratio with 8 KV heads. We find that encoding position with a relative positional embedding<label for="sn-rel-attn-shaw" class="sidenote-number"></label><input type="checkbox" id="sn-rel-attn-shaw" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/1803.02155">Self-Attention with Relative Position Representations</a> (Peter Shaw et al, 2018)</span><label for="sn-rel-attn-music-transformer" class="sidenote-number"></label><input type="checkbox" id="sn-rel-attn-music-transformer" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/1809.04281">Music Transformer</a> (Cheng-Zhi Anna Huang et al, 2018)</span> performs better and extrapolates better to longer sequences than the more widely adopted Rotary Positional Embedding (RoPE). We also apply short convolutions at two points — after the key and value projections in each attention layer, and on the attention and MLP residual branch outputs before they rejoin the main residual stream.</p> <h3 id="training">Training</h3> <p>Inkling was pretrained on 45 trillion tokens from a variety of content types, including text, images, audio and video. We trained Inkling with a hybrid optimization strategy — Muon for large matrix weights, Adam for other parameters — and hyperparameter schedules inspired by our previous research on <a href="https://thinkingmachines.ai/blog/modular-manifolds/">modular manifolds</a>. We coupled the weight decay strength to the square of the learning rate, which we found kept the overall size of the model weights stable across training horizons<label for="sn-weight-decay" class="sidenote-number"></label><input type="checkbox" id="sn-weight-decay" class="margin-toggle"><span class="sidenote">See also <a href="https://arxiv.org/abs/2305.17212">Kosson et al. (2023)</a> and <a href="https://arxiv.org/abs/2506.02285v1">Defazio (2025)</a>.</span>.</p> <p>We post-trained Inkling on a broad distribution of math, agentic code &amp; tool use, audio, image, chat, and safety domains. To bootstrap post-training, we ran an initial SFT on synthetic data generated by open-weights models including Kimi K2.5. The bootstrap accounts for a small fraction of compute, with the majority being employed for large-scale RL on synthetic and human-created environments.</p> <p>Inkling was our first major training effort and was trained on NVIDIA GB300 NVL72 systems. Future models will further push the scale of compute across pre-training, post-training and RL.</p> <h3 id="rl-at-scale">RL at scale</h3> <p>We relied on large-scale asynchronous RL to shape model behavior and improve its reasoning and overall performance. The chart below shows the model’s score on a held-out aggregate of reasoning evals such as AIME, HLE, GPQA, and others. We scaled RL to over 30M rollouts, with stable training sustained over two long continuous runs. Reasoning performance improved log-linearly throughout the entire process, resulting in a significant increase overall.</p> <figure> <div class="rl-scaling-figure"> <div class="rl-scaling"> <div class="benchmark-results-controls chart-hscroll-controls" role="group" aria-label="RL scaling chart controls"> <button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll RL scaling chart left" title="Scroll left"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg> </button> <button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll RL scaling chart right" title="Scroll right"> <svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg> </button> </div> <div class="rl-scaling-scroll chart-hscroll" role="img" aria-label="A line chart of aggregate eval reward against RL rollouts on a logarithmic x-axis from zero to over thirty million. Reward rises log-linearly from 0.264 at SFT initialization to 0.356 at the released checkpoint (about 26.5 million rollouts), with the checkpoints falling close to a straight-line fit."> <svg viewBox="0 0 760 400" role="img" aria-hidden="true" focusable="false"> <g class="rl-scaling-legend" transform="translate(84 0)"> <g transform="translate(78 30)"><circle class="rl-scaling-dot-sft" cx="0" cy="0" r="6"></circle><text x="14" y="4">SFT initialization</text></g> <g transform="translate(240 30)"><circle class="rl-scaling-dot-rl" cx="0" cy="0" r="6"></circle><text x="14" y="4">RL checkpoint</text></g> <g transform="translate(388 30)"><circle class="rl-scaling-dot-rel" cx="0" cy="0" r="6"></circle><text x="14" y="4">Released checkpoint</text></g> </g> <g class="rl-scaling-panel"> <line class="rl-scaling-grid" x1="78" y1="320.0" x2="724" y2="320.0"></line> <text class="rl-scaling-ytick" x="66" y="324.0">0.25</text> <line class="rl-scaling-grid" x1="78" y1="257.5" x2="724" y2="257.5"></line> <text class="rl-scaling-ytick" x="66" y="261.5">0.28</text> <line class="rl-scaling-grid" x1="78" y1="195.0" x2="724" y2="195.0"></line> <text class="rl-scaling-ytick" x="66" y="199.0">0.31</text> <line class="rl-scaling-grid" x1="78" y1="132.5" x2="724" y2="132.5"></line> <text class="rl-scaling-ytick" x="66" y="136.5">0.34</text> <line class="rl-scaling-grid" x1="78" y1="70.0" x2="724" y2="70.0"></line> <text class="rl-scaling-ytick" x="66" y="74.0">0.37</text> <line class="rl-scaling-axis" x1="78" y1="70" x2="78" y2="320"></line> <line class="rl-scaling-axis" x1="78" y1="320" x2="724" y2="320"></line> <line class="rl-scaling-axis" x1="110.0" y1="320" x2="110.0" y2="325"></line> <text class="rl-scaling-xtick" x="110.0" y="340">0</text> <line class="rl-scaling-axis" x1="179.7" y1="320" x2="179.7" y2="325"></line> <text class="rl-scaling-xtick" x="179.7" y="340">0.5M</text> <line class="rl-scaling-axis" x1="229.1" y1="320" x2="229.1" y2="325"></line> <text class="rl-scaling-xtick" x="229.1" y="340">1M</text> <line class="rl-scaling-axis" x1="298.8" y1="320" x2="298.8" y2="325"></line> <text class="rl-scaling-xtick" x="298.8" y="340">2M</text> <line class="rl-scaling-axis" x1="386.5" y1="320" x2="386.5" y2="325"></line> <text class="rl-scaling-xtick" x="386.5" y="340">4M</text> <line class="rl-scaling-axis" x1="487.5" y1="320" x2="487.5" y2="325"></line> <text class="rl-scaling-xtick" x="487.5" y="340">8M</text> <line class="rl-scaling-axis" x1="596.8" y1="320" x2="596.8" y2="325"></line> <text class="rl-scaling-xtick" x="596.8" y="340">16M</text> <line class="rl-scaling-axis" x1="700.0" y1="320" x2="700.0" y2="325"></line> <text class="rl-scaling-xtick" x="700.0" y="340">30M+</text> <line class="rl-scaling-fit" x1="110.0" y1="308.4" x2="679.2" y2="106.1"></line> <circle class="rl-scaling-dot-rl" cx="188.1" cy="296.4" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="287.4" cy="242.4" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="452.3" cy="204.8" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="493.2" cy="175.9" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="514.6" cy="160.8" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="557.1" cy="143.1" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="597.1" cy="136.3" r="5"></circle> <circle class="rl-scaling-dot-rl" cx="634.2" cy="122.1" r="5"></circle> <circle class="rl-scaling-dot-sft" cx="110.0" cy="290.6" r="6.5"></circle> <text class="rl-scaling-anno" x="121.0" y="281.6">SFT init · 0.264</text> <circle class="rl-scaling-dot-rel" cx="679.2" cy="98.3" r="7.5"></circle> <text class="rl-scaling-anno rl-scaling-anno-end" x="668.2" y="87.3">Released · 0.356</text> <text class="rl-scaling-axis-label" x="26" y="195.0" transform="rotate(-90 26 195.0)">Aggregate eval reward</text> </g> <text class="rl-scaling-axis-label rl-scaling-xlabel" x="401.0" y="372">Rollouts →</text> </svg> </div> </div> </div> <figcaption>Reward on a held-out aggregate of reasoning evals — AIME, HLE, GPQA, and others — improves log-linearly over more than 30M RL rollouts, from SFT initialization to the released checkpoint.</figcaption></figure> <p>We specified the model’s effort level on different samples by changing the system message and adjusting the per-token cost. This caused the model to use a different amount of tokens in different rollouts and learn the ability to control thinking effort.</p> <p>We also observed an emergent shift in the reasoning style over the course of RL training. The chain of thought became more concise over time, dropping grammatical overhead while remaining comprehensible and leaving the final response unaffected. This wasn’t targeted by the reward — efficiency alone drove the compression. A similar effect was also recently noted by the Cognition team in the process of training SWE-1.7<label for="sn-swe17" class="sidenote-number"></label><input type="checkbox" id="sn-swe17" class="margin-toggle"><span class="sidenote">The Cognition Team, “<a href="https://cognition.com/blog/swe-1-7">SWE-1.7: Frontier Intelligence at a Fraction of the Cost</a>.”</span>. Below is an example of how Inkling’s chain of thought on the same math problem evolved with RL:</p> <figure> <div class="cot-evo-figure"> <div class="cot-evo" role="img" aria-label="Two side-by-side excerpts of the model's chain of thought on the same physics problem, early versus late in RL. The early trace uses full, grammatical sentences; the late trace is compressed and telegraphic — dropping articles and connectives while staying comprehensible."> <div class="cot-evo-cols"> <div class="cot-evo-col cot-evo-col-early"> <div class="cot-evo-label">Early in RL <span class="cot-evo-tokens">verbose, grammatical</span></div> <div class="cot-evo-trace"><strong>We need to understand</strong> the operator. The 5D line element is ds² = e^{2A(x)} (ds²_4d + dx²), where A(x) = sin(x) + 4 cos(x), x in [0, 2π]. The internal coordinate is periodic. The background is a warped product: metric g_{MN} where M,N = 0..4. The internal direction has metric e^{2A(x)} dx²? Wait, the ds² is e^{2A} (ds²_4d + dx²). So the internal metric is e^{2A(x)} dx². Actually if the total metric is ds² = e^{2A(x)} (ds²_4d + dx²), then yes, internal metric is e^{2A} dx². <span class="cot-evo-ellipsis">…</span></div> </div> <div class="cot-evo-col cot-evo-col-late"> <div class="cot-evo-label">Late in RL <span class="cot-evo-tokens">compressed, telegraphic</span></div> <div class="cot-evo-trace"><strong>We need determine eigenvalue problem</strong> for spin-2 fluctuations h_{μν}(x,y) with TT in 4d and depend on x. For metric of form ds² = e^{2A(x)} (g_{μν}(y) + h_{μν}(y,x)) dy^μ dy^ν + e^{2A(x)}? Wait internal metric is e^{2A} dx²? Actually ds² = e^{2A} [ds_4² + dx²]. So internal metric is e^{2A} dx²; warp factor same for 4d and internal? Yes. We need equation for h_{μν}(y,x) = h_{μν}(y) ψ(x) maybe with normalization. <span class="cot-evo-ellipsis">…</span></div> </div> </div> </div> </div> <figcaption>The same chain of thought, early and late in RL. The late-RL trace drops articles and connectives — “We need <em>to</em> understand” becomes “We need determine” — while staying comprehensible and reaching the same answer.</figcaption></figure> <h2 id="inkling-small">Inkling-Small</h2> <p>Alongside Inkling we are sharing a preview of <a href="https://thinkingmachines.ai/news/inkling-small/">Inkling-Small</a><label for="sn-inkling-small-launch" class="sidenote-number"></label><input type="checkbox" id="sn-inkling-small-launch" class="margin-toggle"><span class="sidenote">Update: Two weeks after this post was published, we launched <a href="https://thinkingmachines.ai/news/inkling-small/">Inkling-Small</a>.</span>, a 276B-parameter Mixture-of-Experts model (12B active, vs. 41B for Inkling) with a different performance/latency trade-off. Inkling-Small matches or exceeds its larger sibling on many benchmarks — the result of improvements we made to the pre-training data and recipe for the smaller model. The two models share the same scalable post-training stack applied on top.</p> <div class="inkling-eval-table-block" style="--benchmark-model-count: 2"><div class="benchmark-results-controls inkling-eval-table-controls" role="group" aria-label="Inkling-Small comparison table controls"><button class="benchmark-results-control" type="button" data-benchmark-scroll="left" aria-label="Scroll Inkling-Small comparison table left" title="Scroll table left"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M15 18 L9 12 L15 6"></path></svg></button><button class="benchmark-results-control" type="button" data-benchmark-scroll="right" aria-label="Scroll Inkling-Small comparison table right" title="Scroll table right"><svg class="benchmark-results-control-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false"><path d="M9 6 L15 12 L9 18"></path></svg></button></div><div class="benchmark-results-scroll inkling-eval-table inkling-eval-table--content" data-inkling-eval-table=""><table class="benchmark-results-table benchmark-results-categorized"><colgroup><col class="benchmark-name-column"><col class="benchmark-model-column" span="2"></colgroup><thead><tr class="benchmark-model-header-row"><td class="benchmark-heading-spacer" aria-hidden="true"></td><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling</span></span><span class="benchmark-subtitle">effort=0.99</span></th><th class="benchmark-model"><span class="benchmark-model-name"><span class="benchmark-model-name-line">Inkling-Small</span><span class="benchmark-model-name-line">(Preview)</span></span><span class="benchmark-subtitle">effort=0.99</span></th></tr></thead><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Reasoning</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HLE</span><span class="benchmark-subtitle">text only</span></td><td class="benchmark-value benchmark-value-best">29.7%</td><td class="benchmark-value">29.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">HLE</span><span class="benchmark-subtitle">with tools</span></td><td class="benchmark-value">46.0%</td><td class="benchmark-value benchmark-value-best">46.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">AIME 2026</span></td><td class="benchmark-value benchmark-value-best">97.1%</td><td class="benchmark-value">95.1%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">GPQA Diamond</span></td><td class="benchmark-value">87.2%</td><td class="benchmark-value benchmark-value-best">88.3%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Agentic (coding)</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SWEBench Verified</span></td><td class="benchmark-value benchmark-value-best">77.6%</td><td class="benchmark-value">77.4%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SWEBench Pro</span><span class="benchmark-subtitle">Public</span></td><td class="benchmark-value benchmark-value-best">54.3%</td><td class="benchmark-value">53.2%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Terminal Bench 2.1</span><span class="benchmark-subtitle">Best Harness</span></td><td class="benchmark-value benchmark-value-best">63.8%*</td><td class="benchmark-value">52.7%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Agentic (general)</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Tau 3 Banking</span></td><td class="benchmark-value benchmark-value-best">23.7%</td><td class="benchmark-value">13.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MCP-Atlas</span></td><td class="benchmark-value benchmark-value-best">76.0%</td><td class="benchmark-value">74.9%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Factuality</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">SimpleQA Verified</span></td><td class="benchmark-value benchmark-value-best">43.9%</td><td class="benchmark-value">20.9%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Chat</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">IFBench</span></td><td class="benchmark-value">79.8%</td><td class="benchmark-value benchmark-value-best">83.4%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Global-MMLU-Lite</span></td><td class="benchmark-value benchmark-value-best">88.7%</td><td class="benchmark-value">86.8%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Vision</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMMU Pro</span><span class="benchmark-subtitle">Standard 10</span></td><td class="benchmark-value benchmark-value-best">73.5%</td><td class="benchmark-value">73.1%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span></td><td class="benchmark-value benchmark-value-best">78.1%</td><td class="benchmark-value">76.7%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Charxiv RQ</span><span class="benchmark-subtitle">with python</span></td><td class="benchmark-value">82.0%</td><td class="benchmark-value benchmark-value-best">83.4%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Audio</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">Audio MC</span></td><td class="benchmark-value benchmark-value-best">56.6%</td><td class="benchmark-value">49.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">MMAU</span></td><td class="benchmark-value">77.2%</td><td class="benchmark-value benchmark-value-best">77.5%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">VoiceBench</span></td><td class="benchmark-value benchmark-value-best">91.4%</td><td class="benchmark-value">90.0%</td></tr></tbody><tbody class="benchmark-category-group"><tr class="benchmark-category-row"><th class="benchmark-category-label" scope="rowgroup"><span class="benchmark-category-label-text">Safety</span></th><td class="benchmark-category-cell" aria-hidden="true"></td><td class="benchmark-category-cell" aria-hidden="true"></td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">Adversarial</span></td><td class="benchmark-value benchmark-value-best">78.0%</td><td class="benchmark-value">75.6%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">FORTRESS</span><span class="benchmark-subtitle">Benign</span></td><td class="benchmark-value benchmark-value-best">95.9%</td><td class="benchmark-value">94.1%</td></tr><tr><td class="benchmark-name"><span class="benchmark-title">StrongREJECT</span></td><td class="benchmark-value">98.6%</td><td class="benchmark-value benchmark-value-best">98.8%</td></tr></tbody></table></div></div> <p class="benchmark-results-note">Both models reported at effort=0.99; the higher result in each row is highlighted. *We assign a score of 0 to Terminal Bench 2.1 rollouts with solution contamination from web search.</p> <p>Early results show Inkling-Small performing close to Inkling on reasoning and agentic tasks. With 12B active parameters and controllable thinking effort, it is a natural fit for workloads where cost and latency matter such as coding, using LLMs to grade, or generating synthetic data for other models.</p> <p>We are currently finishing the testing of Inkling-Small and will release its full weights once that work is complete.</p> <h2 id="customizing-inkling">Customizing Inkling</h2> <p><a href="https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/">Many real-world problems</a> aren’t solved well by even the best generalist models, with the gap being closed by fine-tuning that utilizes an organization’s specialized knowledge. The experience of our Tinker customers points in the same direction. Our post-training and results of RL at scale suggest that Inkling is capable of rapidly learning from fine-tuning.</p> <h3 id="inkling-availability">Inkling availability</h3> <p>Inkling is available on <a href="https://thinkingmachines.ai/tinker/">Tinker</a> today with context length options of 64K and 256K tokens. We are offering Inkling at a 50% discount for a limited time, with full pricing information available <a href="https://tinker-docs.thinkingmachines.ai/tinker/models/">in our documentation</a>.</p> <p>To support Tinkerers fine-tuning with Inkling, we have updated our cookbook to natively support Inkling and have added three <a href="https://github.com/thinking-machines-lab/tinker-cookbook">new cookbook recipes</a> that showcase Inkling’s unique audio capabilities. We also released <a href="https://pypi.org/project/tml-renderers/">tml-renderer</a> for reliably sampling and post-training with tool calls, reasoning content, and multimodal inputs.</p> <p>To get a feel for the model before committing to a run, users can head to the <a href="https://tinker.thinkingmachines.ai/playground?utm_source=blog&amp;utm_campaign=inkling_model_release">Inkling Playground</a> in the Tinker console. The playground offers a chat interface with integrated agentic web search, free for a limited time.</p> <p>We have partnered across the ecosystem to help customers deploy checkpoints fine-tuned on Tinker. Inkling is available via APIs on <a href="https://www.together.ai/blog/together-ai-brings-thinking-machines-labs-new-model-inkling-on-day-0">Together AI</a>, <a href="https://fireworks.ai/">Fireworks</a>, <a href="https://modal.com/">Modal</a>, <a href="https://www.databricks.com/">Databricks</a>, and <a href="https://www.baseten.co/">Baseten</a>. We worked with <a href="https://www.radixark.com/">RadixArk</a> to provide open-source inference and RL support in <a href="https://www.sglang.io/">SGLang</a> and <a href="https://github.com/radixark/miles">Miles</a>. We worked with <a href="https://inferact.ai/">Inferact</a> to support inference in <a href="https://vllm.ai/">vLLM</a>, with <a href="https://lightseek.org/">Lightseek</a> for inference in <a href="https://github.com/lightseekorg/tokenspeed">TokenSpeed</a>, and with <a href="https://unsloth.ai/">Unsloth</a> for inference in <a href="https://github.com/ggml-org/llama.cpp">llama.cpp</a>. Finally, we partnered with <a href="https://huggingface.co/">Hugging Face</a> on integration with <a href="https://github.com/huggingface/transformers">transformers</a>.</p> <p>Inkling’s full weights are on <a href="https://huggingface.co/thinkingmachines/inkling">Hugging Face</a>, both as the original checkpoint and as an NVFP4 checkpoint for efficient inference on NVIDIA Blackwell systems.</p></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/introducing-inkling/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Learning to Replicate Expert Judgment in Financial Tasks

<div class="post-cover"><figure class="judgment-map-figure" aria-label="Animated field of model-like numeric weights becoming structured after fine-tuning."> <div class="judgment-map" role="img" aria-label="A rectangular field of gray digits where random numbers move through low-value regions while higher-value regions hold organized numeric contour patterns, representing expert-labeled fine-tuning shaping a model's internal judgments."> <img class="judgment-map-fallback" src="https://imgproxy.citrusreader.com/qdoT9jNzzZPS2CvsvQgvoh2dJnKtbLeVi0by5fpIUgA/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvbGVhcm5pbmctdG8tcmVwbGljYXRlLWV4cGVydC1qdWRnbWVudC1pbi1maW5hbmNpYWwtdGFza3MvaW1hZ2VzL2p1ZGdtZW50LW1hcC1mYWxsYmFjay5zdmc" alt aria-hidden="true" loading="eager" decoding="async" referrerpolicy="no-referrer"> <canvas class="judgment-map-canvas" aria-hidden="true"></canvas> </div> </figure> </div><div class="post-content-shell"><article class="content"><h2 id="judging-information">Judging information</h2> <p>Outperforming the market is hard. When every investor has access to the same sources of public information, alpha must come from unique insight built on taste and judgment. A strong investor’s judgment is difficult to articulate and teach directly to others, whether human or AI. It comes from experience.</p> <p>Even when we decompose an investor’s job into its simplest constituent tasks, those tasks turn out to be surprisingly difficult for LLMs. In this post, we consider a simple special case: filtering and processing financial documents to surface information relevant to investment decisions.</p> <p>Investors are bombarded with information every day: news articles, research reports, company documents, emails, internal write-ups, and more. Reading is the easy part. The real work is the small, repeated judgments carried over it — filtering, interpreting, segmenting, and identifying where the useful signal lies. These judgments are embedded throughout an investor’s daily workflow and consume substantial time.</p> <p>We wanted to see if we could automate the information triage task: identifying what is relevant and interesting to read. This alone could greatly augment investors’ productivity, letting them spend their freed up attention on higher-level synthesis and decision making.</p> <p>Given that LLMs perform poorly on simple financial tasks, we asked: is it possible to teach LLMs financial judgement? We find that with <strong>high-quality human annotations</strong>, we can teach LLMs to interpret text with expert-level taste and judgement. <strong>Our proprietary model outperforms all frontier models we tested on information accuracy and recall, at a fraction of their cost.</strong></p> <p>We describe our training process and results on a subset of data cleared for public release. Based on our results, we further describe the seeds of a vision of <em>differentiated intelligence</em>, with models tuned for specific organizational needs.</p> <h2 id="frontier-model-performance">Frontier model performance</h2> <p>We evaluated models on six information filtering tasks drawn from investors’ daily workflows. Beyond these tasks, we have many others internally that show similar patterns to these six tasks: frontier models we tested on underperform compared to our internally trained models.</p> <p>We measured accuracy — the percentage of documents that were correctly labeled according to our investors. For classification tasks, we also calculated the F1 score.<label for="sn-judging-1" class="sidenote-number"></label><input type="checkbox" id="sn-judging-1" class="margin-toggle"><span class="sidenote"><a href="https://en.wikipedia.org/wiki/F-score" target="_blank" rel="noopener">F-score</a> (Wikipedia).</span></p> <figure id="fig:financial-tasks"><div class="judging-task-cards" role="list" aria-label="Six financial tasks evaluated in this post."><article class="judging-task-card" role="listitem"><div class="judging-task-card-heading"><span class="judging-task-card-number">01</span><p class="judging-task-card-title">Financial Article Relevancy</p></div><p class="judging-task-card-description">Given a financial article, classify whether it is relevant to a C-suite investment professional.</p><p class="judging-task-card-metrics-label">EVAL METRICS</p><p class="judging-task-card-metrics">F1 score, Accuracy</p></article><article class="judging-task-card" role="listitem"><div class="judging-task-card-heading"><span class="judging-task-card-number">02</span><p class="judging-task-card-title">Central Bank Document Relevancy</p></div><p class="judging-task-card-description">Given a central bank document, classify whether it signals the direction of future interest rate changes.</p><p class="judging-task-card-metrics-label">EVAL METRICS</p><p class="judging-task-card-metrics">F1 score, Accuracy</p></article><article class="judging-task-card" role="listitem"><div class="judging-task-card-heading"><span class="judging-task-card-number">03</span><p class="judging-task-card-title">Generic Document Relevancy</p></div><p class="judging-task-card-description">Given an investor’s question and a research document, classify whether the document helps answer it.</p><p class="judging-task-card-metrics-label">EVAL METRICS</p><p class="judging-task-card-metrics">F1 score, Accuracy</p></article><article class="judging-task-card" role="listitem"><div class="judging-task-card-heading"><span class="judging-task-card-number">04</span><p class="judging-task-card-title">Ad Hoc Content Labeling</p></div><p class="judging-task-card-description">Research documents are either recurring (repeated boilerplate) or mixed (boilerplate plus one-off, issue-specific analysis). Classify which, and find the last page of issue-specific content.</p><p class="judging-task-card-metrics-label">EVAL METRICS</p><p class="judging-task-card-metrics">Accuracy</p></article><article class="judging-task-card" role="listitem"><div class="judging-task-card-heading"><span class="judging-task-card-number">05</span><p class="judging-task-card-title">Document Truncation</p></div><p class="judging-task-card-description">Identify where boilerplate content begins in a document.</p><p class="judging-task-card-metrics-label">EVAL METRICS</p><p class="judging-task-card-metrics">Exact Match Accuracy</p></article><article class="judging-task-card" role="listitem"><div class="judging-task-card-heading"><span class="judging-task-card-number">06</span><p class="judging-task-card-title">Email Truncation</p></div><p class="judging-task-card-description">Identify where boilerplate content begins in an email.</p><p class="judging-task-card-metrics-label">EVAL METRICS</p><p class="judging-task-card-metrics">Exact Match Accuracy</p></article></div><figcaption>The six financial tasks we evaluate in this blog post, each drawn from the routine work of an investor.</figcaption></figure> <p>These tasks are trivial for investors, but they get stuck when articulating their decision process. Consider the following example of classifying a news article as relevant to an investment professional below:</p> <figure id="fig:article-relevance-example" class="ft-relevance-examples-figure"> <div class="ft-relevance-examples" role="group" aria-label="Two financial article examples classified by relevance."> <article class="ft-relevance-example-card"> <p class="ft-relevance-example-label">Not relevant</p> <div class="ft-relevance-example-paper"> <a class="ft-relevance-example-source" href="https://www.ft.com/content/e021f12b-d2a0-4637-ac0f-4e3dae5a8843" target="_blank" rel="noopener">ft.com</a> <p class="ft-relevance-example-title">Trump insists Greenland is his</p> <img src="https://imgproxy.citrusreader.com/S1cF757Z5qpFemxByQbZrpIaLONI_-utM-T1m6lSwlA/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvbGVhcm5pbmctdG8tcmVwbGljYXRlLWV4cGVydC1qdWRnbWVudC1pbi1maW5hbmNpYWwtdGFza3MvaW1hZ2VzL2Z0LTEuanBn" alt="Illustration from an article about Trump and Greenland." loading="lazy" decoding="async" referrerpolicy="no-referrer"> <p class="ft-relevance-example-deck">© Jeremy Banx</p> </div> </article> <article class="ft-relevance-example-card"> <p class="ft-relevance-example-label">Relevant</p> <div class="ft-relevance-example-paper"> <a class="ft-relevance-example-source" href="https://www.ft.com/content/b9ae2417-2e89-4b0a-bad5-d94f4e980ecc?syn-25a6b1a6=1" target="_blank" rel="noopener">ft.com</a> <p class="ft-relevance-example-title">US stocks close sharply lower after Trump threatens new China tariffs</p> <img src="https://imgproxy.citrusreader.com/4YFhw-TvDllC88vZ8WtEKi3ARVG3Gv2empoELaMuNFw/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvbGVhcm5pbmctdG8tcmVwbGljYXRlLWV4cGVydC1qdWRnbWVudC1pbi1maW5hbmNpYWwtdGFza3MvaW1hZ2VzL2Z0LTIuanBn" alt="Trader reacting on a market floor during a sharp S&amp;P 500 drop." loading="lazy" decoding="async" referrerpolicy="no-referrer"> <p class="ft-relevance-example-deck">Biggest one-day drop in S&amp;P 500 since April brings weeks long rally to a halt © AFP/Getty Images</p> </div> </article> </div> <figcaption>Example of judging the relevance of a financial article to US markets. Source: Financial Times.</figcaption> </figure> <p>The Greenland example is unlikely to be taken seriously given the context of the article, while the China tariffs are highly relevant. Yet both examples touch on geopolitics and finance.</p> <p>In contrast to our investors, frontier models we tested on perform surprisingly poorly. Variants of Gemini, Claude, and GPT averaged a mere ~50% accuracy when given a prompt that simply states each of the six tasks to perform.</p> <p>We first tried to improve LLM performance with stronger prompting. Our experts wrote instructions based on real task descriptions, and also suggested reframing certain tasks. For example, while an article about a small IPO is clearly financially relevant, it lacks the broad significance that would make it interesting to a macroeconomic investor at Bridgewater. LLM performance on the article classification task improved when they were asked to sort news stories into three labels: relevant and interesting, relevant but uninteresting, and irrelevant.</p> <p>These changes boosted their accuracy from a coin flip to the mid-70s. We saw no further gains in accuracy from automatic prompt-optimization methods. With our best prompts the frontier models we tested on still achieved less than 80% accuracy — the threshold investors expect from a system they could trust in their daily workflow.</p> <figure id="fig:frontier-model-performance" class="prompt-accuracy-bars-figure"> <div class="prompt-accuracy-bars" role="img" aria-label="Grouped bar chart comparing average accuracy across five frontier models with a naive prompt versus an expert-written prompt. Naive prompt accuracies range from 45.6 to 50.1 percent. Expert prompt accuracies range from 74.3 to 78.2 percent."> <div class="prompt-accuracy-bars-legend" aria-hidden="true"> <span class="prompt-accuracy-bars-legend-item"> <span class="prompt-accuracy-bars-legend-swatch prompt-accuracy-bars-legend-swatch-naive"></span> Naive prompt </span> <span class="prompt-accuracy-bars-legend-item"> <span class="prompt-accuracy-bars-legend-swatch prompt-accuracy-bars-legend-swatch-expert"></span> Expert prompt </span> </div> <div class="prompt-accuracy-bars-chart"> <p class="prompt-accuracy-bars-y-label" aria-hidden="true">Average accuracy</p> <div class="prompt-accuracy-bars-plot"> <div class="prompt-accuracy-bars-y-axis" aria-hidden="true"> <span>100%</span> <span>80%</span> <span>60%</span> <span>40%</span> <span>20%</span> <span>0%</span> </div> <div class="prompt-accuracy-bars-plot-area"> <div class="prompt-accuracy-bars-canvas"> <div class="prompt-accuracy-bars-grid" aria-hidden="true"> <span></span> <span></span> <span></span> <span></span> <span></span> <span></span> </div> <div class="prompt-accuracy-bars-groups"> <div class="prompt-accuracy-bars-group" data-model="Claude Opus 4.6" data-accuracy="77.2%" data-f1="85.4%" data-cost="$61.52"> <div class="prompt-accuracy-bars-pair"> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-naive" style="--bar-value: 47.2;"> <span class="prompt-accuracy-bars-bar-value">47.2</span> </div> </div> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-expert" style="--bar-value: 77.2;"> <span class="prompt-accuracy-bars-bar-value">77.2</span> </div> </div> </div> </div> <div class="prompt-accuracy-bars-group" data-model="Gemini 3.1 Pro" data-accuracy="74.3%" data-f1="87.9%" data-cost="$31.77"> <div class="prompt-accuracy-bars-pair"> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-naive" style="--bar-value: 50.1;"> <span class="prompt-accuracy-bars-bar-value">50.1</span> </div> </div> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-expert" style="--bar-value: 74.3;"> <span class="prompt-accuracy-bars-bar-value">74.3</span> </div> </div> </div> </div> <div class="prompt-accuracy-bars-group" data-model="GPT 5.4" data-accuracy="75.8%" data-f1="86.2%" data-cost="$28.50"> <div class="prompt-accuracy-bars-pair"> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-naive" style="--bar-value: 47.2;"> <span class="prompt-accuracy-bars-bar-value">47.2</span> </div> </div> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-expert" style="--bar-value: 75.8;"> <span class="prompt-accuracy-bars-bar-value">75.8</span> </div> </div> </div> </div> <div class="prompt-accuracy-bars-group" data-model="GPT 5.5" data-accuracy="78.2%" data-f1="87.2%" data-cost="$65.06"> <div class="prompt-accuracy-bars-pair"> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-naive" style="--bar-value: 48.5;"> <span class="prompt-accuracy-bars-bar-value">48.5</span> </div> </div> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-expert" style="--bar-value: 78.2;"> <span class="prompt-accuracy-bars-bar-value">78.2</span> </div> </div> </div> </div> <div class="prompt-accuracy-bars-group" data-model="Claude Opus 4.8" data-accuracy="78.0%" data-f1="88.3%" data-cost="$92.59"> <div class="prompt-accuracy-bars-pair"> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-naive" style="--bar-value: 45.6;"> <span class="prompt-accuracy-bars-bar-value">45.6</span> </div> </div> <div class="prompt-accuracy-bars-bar-track"> <div class="prompt-accuracy-bars-bar prompt-accuracy-bars-bar-expert" style="--bar-value: 78;"> <span class="prompt-accuracy-bars-bar-value">78.0</span> </div> </div> </div> </div> </div> </div> <div class="prompt-accuracy-bars-x-labels" aria-hidden="true"> <div class="prompt-accuracy-bars-x-label"> <strong>Opus 4.6</strong> <span>Feb 5</span> </div> <div class="prompt-accuracy-bars-x-label"> <strong>Gemini 3.1 Pro</strong> <span>Feb 19</span> </div> <div class="prompt-accuracy-bars-x-label"> <strong>GPT 5.4</strong> <span>Mar 5</span> </div> <div class="prompt-accuracy-bars-x-label"> <strong>GPT 5.5</strong> <span>Apr 23</span> </div> <div class="prompt-accuracy-bars-x-label"> <strong>Opus 4.8</strong> <span>May 28</span> </div> </div> </div> </div> </div> </div> <figcaption>Accuracy &amp; Positive Class F1 score of frontier models on our financial tasks after manual and automatic prompt engineering. F1 score is averaged across our 3 classification tasks, and accuracy is averaged across all 6 tasks.</figcaption> </figure> <p>Our results also suggest that newer models aren’t improving rapidly at this task, especially per dollar spent. GPT 5.4 costs 43% more than 5.2 but is only marginally more accurate.</p> <p>An explicit prompt can only convey the intuition an expert is able to put into words, while the judgments that matter most are often the hardest to articulate. Fine-tuning sidesteps this: rather than contorting the expert’s intuition into a static prompt, the training process lets the model develop its own judgment. Could we train open-weight models to outperform frontier models we tested on these tasks?</p> <h2 id="training-dataset-construction">Training dataset construction</h2> <p>The first challenge of training a custom model was acquiring a dataset that reflects <strong>high-quality investor taste</strong>. In particular, much of the information is only useful when filtered through an investment professional’s judgment.</p> <p>We initially sourced a dataset from vendors providing non-expert labeling. Models trained on this dataset still performed poorly. After examining the reasoning traces of the model we realized that the labels in the dataset were often wrong. Since expert labelers are costly, we devised a verification scheme that routes only the contested examples to experts.</p> <p>The scheme worked as follows: we trained a model on the dataset from non-expert labelers, then evaluated it on the same data. Examples where the model’s answer differed from the labelers’ were sent to our experts for reevaluation — if a model couldn’t match an example from its own training set then either the example is genuinely difficult, or the original label was wrong. This procedure was used to clean the training set data; the final evaluation was done on a held out test set.</p> <h2 id="training-recipe">Training recipe</h2> <p>We trained our models on Tinker from Thinking Machines Lab.<label for="sn-judging-2" class="sidenote-number"></label><input type="checkbox" id="sn-judging-2" class="margin-toggle"><span class="sidenote"><a href="https://thinkingmachines.ai/tinker/" target="_blank" rel="noopener">Tinker</a>.</span> Tinker allowed us to iterate quickly without worrying about GPU infrastructure.</p> <p>We chose Qwen3-235B as the base model as its fine-tuning performance is widely studied in the academic literature.</p> <p>We began with standard GRPO and importance-sampling loss as a simple, critic-free starting point. This baseline approach resulted in a massive jump in the model performance, but it still fell short of our desired 80% threshold.</p> <table border="1" style="border-collapse: collapse; text-align: center;"><thead><tr><th style="text-align:left;">Model / Training</th><th style="text-align:center;">Average Accuracy</th><th style="text-align:center;">Average Pos F1</th></tr></thead><tbody><tr><td style="text-align:left;">Qwen Base</td><td style="text-align:center;">44.8%</td><td style="text-align:center;">55.24%</td></tr><tr><td style="text-align:left;">Qwen + GRPO</td><td style="text-align:center;">73.48%</td><td style="text-align:center;">88.95%</td></tr></tbody></table> <p>We make the following modifications to our training recipe to push performance farther:</p> <h3 id="1-interleaved-batching">1. Interleaved batching</h3> <p>For our multi-task training recipe, we compared three batching strategies: training each task sequentially, fully mixing tasks within a batch, and interleaving one batch per task in round-robin order. We found interleaving worked best, improving accuracy by 12.1% over fully mixed batches.</p> <h3 id="2-cispo-loss-with-asymmetric-clipping">2. CISPO loss with asymmetric clipping</h3> <p>We used CISPO loss with asymmetric clipping<label for="sn-judging-3" class="sidenote-number"></label><input type="checkbox" id="sn-judging-3" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2510.13786" target="_blank" rel="noopener">CISPO loss with asymmetric clipping</a> (arXiv).</span> to replace the standard importance-sampling loss. Across the loss functions and clipping schemes we tried, this performed best, improving accuracy by 10.1% over the importance-sampling baseline.</p> <h3 id="3-on-policy-distillation-with-strong-teachers">3. On-policy distillation with strong teachers</h3> <p>We train with on-policy distillation<label for="sn-judging-4" class="sidenote-number"></label><input type="checkbox" id="sn-judging-4" class="margin-toggle"><span class="sidenote"><a href="https://thinkingmachines.ai/blog/on-policy-distillation/" target="_blank" rel="noopener">On-Policy Distillation</a>, Kevin Lu in collaboration with others (Thinking Machines).</span> (OPD), constructing the advantage as follows:</p> <span class="katex-display"><span class="katex" tabindex="0"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>r</mi><mo>=</mo><mtext>reward</mtext><mo>−</mo><mi>β</mi><mo>⋅</mo><mi mathvariant="normal">avg</mi><mo>⁡</mo><mo stretchy="false">(</mo><mtext>student_lp</mtext><mo>−</mo><mtext>teacher_lp</mtext><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">r = \text{reward} - \beta \cdot \operatorname{avg}(\text{student\_lp} - \text{teacher\_lp})</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em"></span><span class="mord mathnormal" style="margin-right:0.0278em">r</span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="base"><span class="strut" style="height:0.7778em;vertical-align:-0.0833em"></span><span class="mord text"><span class="mord">reward</span></span><span class="mspace" style="margin-right:0.2222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em"></span></span><span class="base"><span class="strut" style="height:0.8889em;vertical-align:-0.1944em"></span><span class="mord mathnormal" style="margin-right:0.0528em">β</span><span class="mspace" style="margin-right:0.2222em"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em"></span></span><span class="base"><span class="strut" style="height:1.06em;vertical-align:-0.31em"></span><span class="mop"><span class="mord mathrm" style="margin-right:0.0139em">avg</span></span><span class="mopen">(</span><span class="mord text"><span class="mord">student_lp</span></span><span class="mspace" style="margin-right:0.2222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em"></span></span><span class="base"><span class="strut" style="height:1.06em;vertical-align:-0.31em"></span><span class="mord text"><span class="mord">teacher_lp</span></span><span class="mclose">)</span></span></span></span></span> <span class="katex-display"><span class="katex" tabindex="0"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mtext>adv</mtext><mi>i</mi></msub><mo>=</mo><msub><mi>r</mi><mi>i</mi></msub><mo>−</mo><mi mathvariant="normal">avg</mi><mo>⁡</mo><mo stretchy="false">(</mo><mi>r</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\text{adv}_i = r_i - \operatorname{avg}(r)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8444em;vertical-align:-0.15em"></span><span class="mord"><span class="mord text"><span class="mord">adv</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em"><span style="top:-2.55em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="base"><span class="strut" style="height:0.7333em;vertical-align:-0.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0278em">r</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em"></span><span class="mop"><span class="mord mathrm" style="margin-right:0.0139em">avg</span></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.0278em">r</span><span class="mclose">)</span></span></span></span></span> <p>The reward is penalized when the student drifts from the teacher’s distribution, regularizing the policy while it learns the task.</p> <p>Every 20 steps, we promote the current checkpoint to the teacher — but only if validation accuracy has reached a new high, so we never distill toward a weaker model. This gave a further 3.1% gain over a frozen base-model teacher.</p> <h2 id="results">Results</h2> <p>Finding the optimal training recipe required several iterations of different approaches. Tinker’s accessibility allowed us to run fast experiments and refine our approach.</p> <figure class="accuracy-price-scatter-figure"> <div class="accuracy-price-scatter" role="img" aria-label="Scatter plot of average accuracy against cost per 1,000 tasks. Our trained model costs 4.72 dollars and reaches 84.66 percent accuracy, outperforming frontier models that cost between 19.96 and 92.59 dollars and reach between 74.3 and 78.2 percent accuracy."> <svg viewBox="0 0 800 520" aria-hidden="true" focusable="false"> <g class="accuracy-price-scatter-legend"> <g class="accuracy-price-scatter-legend-item" transform="translate(52 34)"> <circle class="accuracy-price-scatter-trained-point" cx="0" cy="0" r="7"></circle> <text x="15" y="4">Our trained model</text> </g> <g class="accuracy-price-scatter-legend-item" transform="translate(223 34)"> <circle class="accuracy-price-scatter-frontier-point" cx="0" cy="0" r="7"></circle> <text x="15" y="4">Frontier models</text> </g> </g> <g class="accuracy-price-scatter-grid"> <line x1="110" y1="70" x2="750" y2="70"></line> <line x1="110" y1="117.9" x2="750" y2="117.9"></line> <line x1="110" y1="165.7" x2="750" y2="165.7"></line> <line x1="110" y1="213.6" x2="750" y2="213.6"></line> <line x1="110" y1="261.4" x2="750" y2="261.4"></line> <line x1="110" y1="309.3" x2="750" y2="309.3"></line> <line x1="110" y1="357.1" x2="750" y2="357.1"></line> <line x1="110" y1="405" x2="750" y2="405"></line> </g> <g class="accuracy-price-scatter-axes"> <line x1="110" y1="70" x2="110" y2="405"></line> <line x1="110" y1="405" x2="750" y2="405"></line> </g> <g class="accuracy-price-scatter-y-ticks"> <text x="90" y="409">72%</text> <text x="90" y="361.1">74%</text> <text x="90" y="313.3">76%</text> <text x="90" y="265.4">78%</text> <text x="90" y="217.6">80%</text> <text x="90" y="169.7">82%</text> <text x="90" y="121.9">84%</text> <text x="90" y="74">86%</text> </g> <g class="accuracy-price-scatter-x-ticks"> <text x="110" y="433">$0</text> <text x="238" y="433">$20</text> <text x="366" y="433">$40</text> <text x="494" y="433">$60</text> <text x="622" y="433">$80</text> <text x="750" y="433">$100</text> </g> <text class="accuracy-price-scatter-y-label" x="41" y="237" transform="rotate(-90 41 237)">Average accuracy</text> <text class="accuracy-price-scatter-x-label" x="430" y="480">Cost per 1,000 tasks (USD)</text> <g class="accuracy-price-scatter-points"> <g class="accuracy-price-scatter-point" data-model="Gemini 3.1 Pro" data-accuracy="74.3%" data-f1="87.9%" data-cost="$31.77"> <circle class="accuracy-price-scatter-frontier-point" cx="313.3" cy="350" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="313.3" cy="350" r="24"></circle> <text class="accuracy-price-scatter-point-label accuracy-price-scatter-point-label-centered" x="313.3" y="377">Gemini 3.1 Pro</text> </g> <g class="accuracy-price-scatter-point" data-model="Claude Opus 4.6" data-accuracy="77.2%" data-f1="85.4%" data-cost="$61.52"> <circle class="accuracy-price-scatter-frontier-point" cx="503.7" cy="280.6" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="503.7" cy="280.6" r="24"></circle> <text class="accuracy-price-scatter-point-label accuracy-price-scatter-point-label-centered" x="503.7" y="307">Claude Opus 4.6</text> </g> <g class="accuracy-price-scatter-point" data-model="Claude Opus 4.8" data-accuracy="78.0%" data-f1="88.3%" data-cost="$92.59"> <circle class="accuracy-price-scatter-frontier-point" cx="702.6" cy="261.4" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="702.6" cy="261.4" r="24"></circle> <text class="accuracy-price-scatter-point-label" x="650" y="247">Claude Opus 4.8</text> </g> <g class="accuracy-price-scatter-point" data-model="GPT 5.2" data-accuracy="74.3%" data-f1="84.6%" data-cost="$19.96"> <circle class="accuracy-price-scatter-frontier-point" cx="237.7" cy="350" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="237.7" cy="350" r="24"></circle> <text class="accuracy-price-scatter-point-label" x="211" y="377">GPT 5.2</text> </g> <g class="accuracy-price-scatter-point" data-model="GPT 5.4" data-accuracy="75.8%" data-f1="86.2%" data-cost="$28.50"> <circle class="accuracy-price-scatter-frontier-point" cx="292.4" cy="314.1" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="292.4" cy="314.1" r="24"></circle> <text class="accuracy-price-scatter-point-label" x="270" y="299">GPT 5.4</text> </g> <g class="accuracy-price-scatter-point" data-model="GPT 5.5" data-accuracy="78.2%" data-f1="87.2%" data-cost="$65.06"> <circle class="accuracy-price-scatter-frontier-point" cx="526.4" cy="256.6" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="526.4" cy="256.6" r="24"></circle> <text class="accuracy-price-scatter-point-label accuracy-price-scatter-point-label-centered" x="526.4" y="244">GPT 5.5</text> </g> <g class="accuracy-price-scatter-point" data-model="Our trained model" data-accuracy="84.66%" data-f1="92.99%" data-cost="$4.72"> <circle class="accuracy-price-scatter-trained-point" cx="140.2" cy="102.8" r="7"></circle> <circle class="accuracy-price-scatter-hit-area" cx="140.2" cy="102.8" r="24"></circle> <text class="accuracy-price-scatter-trained-label" x="154" y="107">Our trained model</text> </g> </g> </svg> </div> <figcaption>Accuracy versus price for our trained model and frontier models. Our model outperforms frontier models on both dimensions across generations.</figcaption> </figure> <p>Our trained model improves average accuracy from 78.2% to 84.7%, meaning the trained model makes 29.8% fewer mistakes than the best frontier model we evaluated. We find this level of accuracy is sufficient for our daily work.</p> <p>Our trained model is also vastly cheaper due to its smaller size: a 13.8x reduction in inference costs per task. As we plan to rely on more models trained to help with specific tasks and to scale AI across the organization, cost is an important consideration.</p> <p>We ablated each part of our training recipe to show how each portion contributes to performance.</p> <figure id="fig:training-ablations"><table border="1" style="border-collapse: collapse; text-align: center;"><thead><tr><th style="text-align:left;">Training Method Ablations</th><th style="text-align:center;">Average Accuracy</th><th style="text-align:center;">Avg Pos F1</th></tr></thead><tbody><tr><td style="text-align:left;">Qwen + Final Recipe</td><td style="text-align:center;">84.66%</td><td style="text-align:center;">92.99%</td></tr><tr><td style="text-align:left;"><span class="judging-ablation-color-square red" tabindex="0" aria-label="Removed individually"></span>Interleaved Batching</td><td style="text-align:center;">72.18%</td><td style="text-align:center;">89.01%</td></tr><tr><td style="text-align:left;"><span class="judging-ablation-color-square red" tabindex="0" aria-label="Removed individually"></span>CISPO + Asymmetric Clips</td><td style="text-align:center;">74.56%</td><td style="text-align:center;">90.64%</td></tr><tr><td style="text-align:left;"><span class="judging-ablation-color-square red" tabindex="0" aria-label="Removed individually"></span>OPD</td><td style="text-align:center;">72.39%</td><td style="text-align:center;">87.93%</td></tr><tr><td style="text-align:left;"><span class="judging-ablation-color-square red" tabindex="0" aria-label="Removed individually"></span>OPD w/ Best Val Accuracy Teacher</td><td style="text-align:center;">81.55%</td><td style="text-align:center;">89.41%</td></tr></tbody></table><figcaption>Each row shows the final recipe with that single component removed (leave one out ablations)</figcaption></figure> <h2 id="conclusion">Conclusion</h2> <p>Frontier models we tested on struggle with relatively simple financial tasks, and model advances don’t improve performance much. In contrast, we’ve shown that <strong>high-quality proprietary datasets</strong> labeled by expert investors and used for fine-tuning produce custom models that exceed frontier performance on our tasks. We have found that this outcome holds true well beyond the six tasks we’ve discussed in this post.</p> <p>Aside from higher accuracy, custom models are also substantially cheaper. We expect to see more productivity gains from custom model training in the future, especially with the availability of training infrastructure like Tinker that enables rapid experimentation.</p> <p>Our results show the possibility of a future of differentiated intelligence, where custom models tuned to specific organizational needs outperform frontier models.</p> <h2 id="citation">Citation</h2> <p>Please cite this work as:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2);overflow-x:auto" tabindex="0" data-language="plaintext"><code><span class="line"><span>Su, Sarah; Zhu, Kevin; Xiao, Emily; Alur, Rohan; Kang, Daniel (Bridgewater AIA Labs), "Learning to replicate expert judgment in financial tasks",</span></span> <span class="line"><span>Thinking Machines Lab: News, June 2026.</span></span></code></pre> <p>Or use the BibTeX citation:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2);overflow-x:auto" tabindex="0" data-language="plaintext"><code><span class="line"><span>@article{su2026expertjudgment,</span></span> <span class="line"><span> author = {Sarah Su, Kevin Zhu, Emily Xiao, Rohan Alur, Daniel Kang (Bridgewater AIA Labs)},</span></span> <span class="line"><span> title = {Learning to replicate expert judgment in financial tasks},</span></span> <span class="line"><span> journal = {Thinking Machines Lab: News},</span></span> <span class="line"><span> year = {2026},</span></span> <span class="line"><span> note = {https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/}</span></span> <span class="line"><span>}</span></span></code></pre></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Announcing Interactivity Research Grants

<div class="post-content-shell"><article class="content"><p>We recently published a research preview of <a href="https://thinkingmachines.ai/blog/interaction-models/">interaction models</a>, an approach to AI designed for human collaboration. Collaboration requires interactivity — the capacity of a model to communicate in real time in different modalities in a way that is intuitive for people.</p> <p>Most AI research in recent years has focused on intelligence and autonomy, with interactivity relegated to secondary importance. We aim to change that — with your help. We are awarding multiple grants of $100,000 (with an additional $25,000 in Tinker credits) for proposals on research that advances interactivity. We encourage you to <a href="https://thinkingmachines.ai/news/interactivity-research-grants/apply/">apply</a> by June 19, 2026 to <a href="https://thinkingmachines.ai/cdn-cgi/l/email-protection#157267747b616655617d7c7b7e7c7b727874767d7c7b70663b747c"><span class="__cf_email__" data-cfemail="1c7b6e7d72686f5c687475727775727b717d7f747572796f327d75">[email&nbsp;protected]</span></a>.</p> <h2 id="research-direction">Research direction</h2> <p>We are looking for research that lays the foundation for interactivity to scale along with intelligence. There are many potential avenues of interactivity research to pursue, here are some that we are excited about:</p> <ul> <li><strong>Evaluations for real-time multimodal interaction.</strong> How do we benchmark the real-time interaction ability of AI systems in real world use cases? The open research community is uniquely positioned to contribute independent, fair evaluations that push this work forward.</li> <li><strong>Ensuring safety for multimodal real-time AI systems.</strong> How do we detect dangerous requests and behaviors from users across modalities and respond in time? These problems pose unique challenges distinct from turn-based text-only safety.</li> <li><strong>Explaining complex information with generative UI.</strong> As AI systems perform increasingly complex tasks, it is important that they can explain their results in a variety of output formats such as SVG or HTML. How do we enable multimodal AI systems to explain their work so that people understand it?</li> <li><strong>Steering agents during long-horizon tasks.</strong> Current agents are optimized to perform long-horizon tasks autonomously without incorporating human feedback. How can humans effectively obtain real-time updates on the agents’ execution and steer it live? We are interested in training approaches and techniques for this capability as well as evals.</li> </ul> <p>We are not limiting our grants to these specific directions. We welcome proposals on any topic that advances this space.</p> <p>See <a href="https://thinkingmachines.ai/news/interactivity-research-grants/apply/">how to apply</a> for proposal requirements, selection criteria, timeline, and complete <a href="https://thinkingmachines.ai/legal/interactivity-research-grants-terms/">terms of service</a> for the grant program.</p> <p>Research on human-AI interactivity is still an emerging field, but we believe in its potential to shape the trajectory of AI development. We’re excited to support this research and look forward to seeing your ideas.</p></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/interactivity-research-grants/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Training LLMs to Predict World Events (Guest Post with Mantic)

<div class="post-cover"><img src="https://imgproxy.citrusreader.com/HdOXWtnhJ5FOD1uy_iih26YQwU99UOPRfajip95wHcQ/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2NvdmVyLnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"></div><div class="post-content-shell"><article class="content"><p><em><a href="https://www.mantic.com/">Mantic</a> have been using Tinker since it launched. This guest post is a technical deep dive on what they have built so far.</em></p> <p>The top AI forecasting systems are approaching superforecaster-level accuracy on geopolitics and current affairs.<label for="sn1" class="sidenote-number"></label><input type="checkbox" id="sn1" class="margin-toggle"><span class="sidenote"><a href="https://news.polymarket.com/p/its-over">It’s over</a> (Polymarket, 2026)</span> This is exciting because scalable, automated forecasting could significantly improve the quality of decision-making across the economy and in government.</p> <p>To date, the most successful recipe in forecasting tournaments has been to combine an off-the-shelf LLM (like Gemini 3 or GPT-5) with forecasting-specific context-gathering. These models, to our knowledge, have not been explicitly trained for forecasting. Can we improve the recipe by replacing them with models fine-tuned specifically for forecasting?</p> <p>We target “judgmental forecasting”: prediction problems that require human-like research and reasoning. Judgmental forecasting is needed for domains like geopolitics, politics, technology, business, and economic policy, where there often isn’t enough data for a standard statistical approach like time-series extrapolation. It was popularized by the book <a href="https://books.google.com/books?hl=en&amp;lr=&amp;id=_lMPDAAAQBAJ&amp;oi=fnd&amp;pg=PA1&amp;ots=5avwaBT55R&amp;sig=B2i7FYispHz5PtB6KVu2Ndt58cE">Superforecasting</a>, and now prediction markets like Polymarket and Kalshi.</p> <p>In this post, we show it’s possible to significantly improve the forecasting performance of gpt-oss-120b using reinforcement learning. With Tinker, we fine-tune a model on around 10,000 binary questions of the form <em>“Will [event] occur before [date]?”</em>. We reward the model for putting greater probability on the correct real-world outcome.</p> <p>In a head-to-head contest, the fine-tuned model achieves marginally superior performance to the frontier LLMs (see Figure 1), despite much lower initial performance. We find that providing forecast-specific context increases the gains from fine-tuning.</p> <p>In the optimal <em>ensemble</em> of different models (which outperforms any single model), Grok 4 and our fine-tuned model are the most important contributors. The fine-tuned model learns a forecasting policy that is as accurate as the frontier LLMs, yet decorrelated from them.</p> <p>Together, the results demonstrate that on-task training can extend the state-of-the-art in AI forecasting.</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/KJewKCgwsHrQltA1S9uzoNbsLcUiUgg3qXO3ieOrtp8/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV8xLnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>RL fine-tuning makes gpt-oss-120b competitive with the frontier LLMs on questions from the Metaculus AI Benchmark Q2 2025. Naively predicting 50% on every question would get a score of 0, and perfect foresight would get a score of 100, per the construction of the Metaculus “baseline score”. Naively predicting 18.8% on every question (the rate at which the equivalent questions resolved “yes” in the previous tournament, Q1 2025) yields a score of 22.3, which we use to truncate the Y-axis. Fine-tuning improves gpt-oss-120b‘s score from 38.6 to 45.8, on par with the best general models.</figcaption></figure> <h2 id="the-best-existing-recipe-uses-off-the-shelf-llms">The best existing recipe uses off-the-shelf LLMs</h2> <p>The past two years have seen considerable progress in AI judgmental forecasting capabilities. In the Metaculus Cup, a major tournament for amateur and professional forecasters, the best AI systems now rival the top humans (Figure 2).</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/YTCICYkCBKOCUM7lcJm9ykfeVo-bFnbMf2xxuQdSD7M/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV8yLnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Human and AI scores in the Metaculus Cup, a premier forecasting competition. Scores from the top 5 AI forecasters have been steadily improving since they first entered in the Summer of 2024. Mantic first entered in the Summer of 2025 and then in Fall 2025 beat the community prediction and the majority of professional forecasters. These results were without fine-tuning.</figcaption></figure> <p>The trend has been driven by more capable off-the-shelf LLMs, and accelerated by more sophisticated forecasting architectures. Our architecture – which has performed well in recent Metaculus tournaments – consists of two standard phases: (1) a research phase, and (2) a prediction phase (Figure 3).<label for="sn3" class="sidenote-number"></label><input type="checkbox" id="sn3" class="margin-toggle"><span class="sidenote">This two-phase process appears in early work on AI forecasting. See: <a href="https://proceedings.neurips.cc/paper_files/paper/2024/hash/5a5acfd0876c940d81619c1dc60e7748-Abstract-Conference.html">Approaching human-level forecasting with language models</a> (Halawi et al., 2024); <a href="https://arxiv.org/abs/2206.15474">Forecasting Future World Events with Neural Networks</a> (Zou et al, 2022).</span></p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/0_k3J12Gt81_o29qEfDfskXQpBroZlABdNZfOCaO-vw/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV8zLnBuZw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Mantic’s architecture. The research phase takes the forecasting question as input and performs deep research to collect information relevant to the question which goes into the prompt for the prediction LLM. The prediction LLM outputs chain-of-thought reasoning and specifies a probability distribution using specialized tools.</figcaption></figure> <p>The research phase is conducted by deep research agents that collect the context needed to make a good prediction. For example, for the question “Will the United States attack Venezuela before 2026?”, search agents will find information about military buildup in the Caribbean, statements from President Trump, the health of the Venezuelan economy, and so on. The collected research is summarized into a prompt for the prediction phase.</p> <p>The model’s task at the prediction phase is to use our specialized tools to output a probability distribution. In this post, we consider a canonical type of forecasting question: “Will [event] occur before [date]?”. We instruct the LLM to parameterize a mixture model for when the event will next occur – illustrated in Figure 4. The mixture model defines a cumulative distribution function, and from that we can read off the probability of the event occurring before the date specified in the original question.</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/qTTEK7OfXlaFTtIkQU6Hk4X0AP-r5U7uRG0HAwazjeY/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV80LnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Illustrative mixture model. The LLM selects: the number of components in the mixture, their parameters, and their respective weights. The LLM is prompted to select components capturing different scenarios that could lead to the event occurring. The final prediction is a weighted combination of the components.</figcaption></figure> <p>In past tournaments, we’ve used off-the-shelf models as the prediction LLM. Existing literature has shown promising results from RL fine-tuning small models using a simple architecture.<label for="sn4" class="sidenote-number"></label><input type="checkbox" id="sn4" class="margin-toggle"><span class="sidenote"><a href="https://scholar.google.com/citations?view_op=view_citation&amp;hl=en&amp;user=oQuvQ0sAAAAJ&amp;citation_for_view=oQuvQ0sAAAAJ:u5HHmVD_uO8C">Outcome-based Reinforcement Learning to Predict the Future</a> (Turtel et al, 2025)</span> Can we improve frontier AI forecasting systems through on-task fine-tuning?</p> <h2 id="training-details">Training details</h2> <h3 id="datasets">Datasets</h3> <p>We train the prediction LLM on ~10k questions about whether an event will happen by a given date. The questions are from August 2024 to December 2025 and the model’s knowledge cutoff is prior to that, so the resolution is known to us but not to the model. We generated the training set using an LLM pipeline similar to existing work.<label for="sn5" class="sidenote-number"></label><input type="checkbox" id="sn5" class="margin-toggle"><span class="sidenote"><a href="https://arxiv.org/abs/2601.22444">Automating Forecasting Question Generation and Resolution for AI Evaluation</a> (Bosse et al, 2026); <a href="https://arxiv.org/abs/2601.06336">Future-as-Label: Scalable Supervision from Real-World Outcomes</a> (Turtel et al, 2026)</span> Before training, we run the research phase for each question and store static prompts for the prediction LLM.</p> <p>We test on unseen questions from the Q2 2025 Metaculus AI Benchmark Tournament.<label for="sn6" class="sidenote-number"></label><input type="checkbox" id="sn6" class="margin-toggle"><span class="sidenote"> The Fall 2025 iteration would have been a more obvious choice (for being more recent) but contains lower quality questions, indicated by less performance differentiated between strong and weak forecasters.</span> We compare models whose knowledge cutoff is before this tournament’s start date. The full list of questions can be accessed on <a href="https://gist.github.com/enjeeneer/86e24a52e6041a3d78e333bcab16984d">Github</a>.</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/IyVx7_cj_TxFBHDGf_Tj3q3A3TwysZVyQzYlrVWf-ss/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV81LnBuZw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Three example binary event questions from the Metaculus Q2 2025 AI Benchmark.</figcaption></figure> <p>We evaluate using the baseline score, following the Metaculus platform. This is log scoring, i.e. ln(probability assigned to true outcome), rescaled such that 100 is the maximum possible score and 0 is the score for a uniform prediction (in our setting, 50%).</p> <h3 id="implementation">Implementation</h3> <p>We run the experiments on <a href="https://thinkingmachines.ai/tinker/">Tinker</a>. Of the models available through the API, we choose to train <a href="https://huggingface.co/openai/gpt-oss-120b">gpt-oss-120b</a> because of its strong initial performance — second only to <a href="https://huggingface.co/moonshotai/Kimi-K2.5">Kimi K2.5</a> — while being cheaper and faster.</p> <p>We use a standard policy gradient algorithm with <a href="https://arxiv.org/abs/2402.03300">GRPO-style advantage normalisation</a> and <a href="https://arxiv.org/abs/2503.20783">no division by the standard deviation</a>. For rewards we use the Brier score which is <a href="https://sites.stat.washington.edu/raftery/Research/PDF/Gneiting2007jasa.pdf">strictly proper</a>. We found that the Brier score leads to more stable training than the log score, even though the log score is also strictly proper. This could be because the Brier score is bounded in [0, 1] and so produces lower-variance policy gradient estimates.</p> <p>Open-source RL packages often use <a href="https://arxiv.org/abs/2309.06180">vLLM</a> for the sampling backend and <a href="https://arxiv.org/abs/2304.11277">FSDP</a> for the training backend. These can disagree on token probabilities produced by identical policies, which biases policy gradient estimates and destabilises training. We found these discrepancies to be lower on Tinker’s integrated infrastructure, but further mitigate them with <a href="https://fengyao.notion.site/off-policy-rl">an importance sampling correction on the advantages</a>.</p> <p>The Brier score reward function is strictly monotonic in the predicted probability for a fixed outcome, so different rollouts almost always produce different rewards. This makes within-group reward ties extremely unlikely and lets us train with a relatively small group size (8) without needing to break ties or induce variance. We use a batch size of 64, as we find that <a href="https://thinkingmachines.ai/blog/lora/">larger batch sizes tend to destabilise training</a>.</p> <h2 id="results">Results</h2> <h3 id="fine-tuning-elevates-gpt-oss-120b-to-frontier-llm-performance">Fine-tuning elevates gpt-oss-120b to frontier LLM performance</h3> <p>The model’s test set score improves through training (Figure 6), moving from an initial score of 38.6 mean baseline points per question (below all frontier models) to a final score of 45.8 mean baseline points per question (marginally above). This demonstrates that on-task forecasting training can provide a large performance uplift.</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/Ok-Obq8Z5xHUL136XDeR_U09nznOmRbc9aIvGoc9JNE/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV82LnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Test set baseline score of gpt-oss-120b with and without Mantic research and tools. In the model-only setup, test set performance improves, but never reaches the initial score of gpt-oss-120b with Mantic research and tools. With Mantic research and tools, gpt-oss-120b climbs 7 points through training and marginally exceeds the performance of Gemini 3 Pro. Training continued for further steps but performance no longer improved.</figcaption></figure> <p>The LLM fine-tuned without the benefit of our pre-generated research phase, and without the tools to construct mixture models, gains only 3 points from training instead of 7. This suggests these prerequisites positively influence the optimization dynamics, in addition to improving initialization.</p> <h3 id="the-fine-tuned-model-is-an-important-member-of-the-optimal-ensemble">The fine-tuned model is an important member of the optimal ensemble</h3> <p>In human forecasting, there is a well-known “wisdom of the crowd” effect: aggregate forecasts from multiple people often outperforms any one individual. This effect, in part, explains the <a href="https://polymarket.com/accuracy">impressive accuracy of prediction markets</a>. Can we get the same benefit from ensembling the predictions of different LLMs?<label for="sn7" class="sidenote-number"></label><input type="checkbox" id="sn1" class="margin-toggle"><span class="sidenote">See also <a href="https://arxiv.org/abs/2402.19379">Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy</a> (Schoenegger et al, 2024)</span></p> <p>To contribute to an ensemble, an LLM must be sufficiently accurate on its own and also decorrelated from other LLMs in the ensemble. Predictions from most frontier LLMs, while accurate, contribute little diversity to the top-performing model (in our case, Gemini 3 Pro) – Figure 7. Among the frontier LLMs, Grok 4 is the exception: its predictions score well whilst correlating less with other frontier LLMs.</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/gOjwcCQyf44rUDpB7UT6jfRe5759guJgZxyzHzxIPqg/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV83LnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Mean baseline score on binary event questions from the Metaculus Q2 AIB Benchmark with respect to their Jensson Shannon divergence from Gemini 3 Pro for a suite of closed source and open source LLMs. Marker colour indicates each model’s weight in the optimal 5-sample ensemble. The optimal ensemble consists of fine-tuned gpt-oss-120b (40%), Gemini 3 Pro (20%), gpt-5 (20%) and Grok 4 (20%).</figcaption></figure> <p>The optimal ensemble, on a budget of 5 samples, is 2 samples from our fine-tuned gpt-oss-120b plus 1 sample from each of Gemini 3 Pro, Grok 4, and GPT-5. We can test each model’s contribution by removing it, recomputing the optimal ensemble, and seeing how much the score degrades. We find that Grok 4 is the least replaceable, with fine-tuned gpt-oss-120b in second place. Other models can be replaced with little to no performance degradation (Figure 8).</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/DR_FXUbUh5aupCHtNDGsp3fSXTNKX4cvlygq7vgCY78/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV84LnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>When selecting an ensemble of frontier and open-source models, Grok 4 and fine-tuned gpt-oss-120b are the least replaceable. Model replaceability is defined as the reduction in score incurred when removing a model from the optimal ensemble. By definition, if a model is not included in the optimal ensemble,there is no cost to removing it.</figcaption></figure> <p>GPT-5 and Gemini 3 Pro make similar predictions, and thus don’t benefit much from ensembling with each other. Both models improve from mixing their predictions with either Grok 4 or the fine-tuned gpt-oss-120b, and Grok 4 also benefits most from mixing with the fine-tuned model. In either optimal 3-way ensemble from Figure 9, the fine-tuned model gets about half of the total weight.</p> <figure style="margin-top: 2rem; margin-bottom: 2.3rem;"><img data-zoomable="true" src="https://imgproxy.citrusreader.com/LvDRiUzdQQD1ive2TXMEoiXbunF0tMXBp_PIDCd8wAo/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdHJhaW5pbmctbGxtcy10by1wcmVkaWN0LXdvcmxkLWV2ZW50cy9zdmdzL2ZpZ3VyZV85LnBuZw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"><figcaption>Three-way ensembles. Left: The optimal three-way ensemble between the fine-tuned gpt-oss-120b, Gemini 3 Pro and GPT-5 is weighted 56%, 26% and 18% respectively, depicted by the black star. Right: The optimal three-way ensemble between the fine-tuned gpt-oss-120b, Gemini 3 Pro and Grok 4 is weighted 48, 26% and 26% respectively, depicted by the white star.</figcaption></figure> <h2 id="conclusions-and-next-steps">Conclusions and next steps</h2> <p>We have shown that we can elevate the forecasting performance of gpt-oss-120b to match frontier LLMs with RL fine-tuning. This work can be extended in many ways, some of which we have already begun exploring:</p> <ol> <li><strong>Training larger models.</strong> Tinker enables training larger models with higher initial performance than gpt-oss, specifically Kimi K2.5.</li> <li><strong>Training on all question formats.</strong> We are already training models on numerical questions such as economic indicators and multiple choice questions such as election results.</li> <li><strong>Improved question sets.</strong> As the models become stronger forecasters, we need more challenging forecasting questions.</li> <li><strong>Information retrieval inside the loop.</strong> We could give the prediction LLM tools for information retrieval and include this in the training loop.</li> </ol> <h2 id="citation">Citation</h2> <p>Please cite this work as:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2);overflow-x:auto" tabindex="0" data-language="plaintext"><code><span class="line"><span>Jeen, Scott; Aitchison, Matthew; and Mantic, "Training LLMs to Predict World Events", </span></span> <span class="line"><span>Thinking Machines Lab: News, Mar 2026.</span></span></code></pre> <p>Or use the BibTeX citation:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2);overflow-x:auto" tabindex="0" data-language="plaintext"><code><span class="line"><span>@article{scott2026forecasting,</span></span> <span class="line"><span> author = {Scott Jeen, Matthew Aitchison, and Mantic},</span></span> <span class="line"><span> title = {Training LLMs to Predict World Events},</span></span> <span class="line"><span> journal = {Thinking Machines Lab: News},</span></span> <span class="line"><span> year = {2026},</span></span> <span class="line"><span> note = {https://thinkingmachines.ai/news/training-llms-to-predict-world-events/}</span></span> <span class="line"><span>}</span></span></code></pre></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/training-llms-to-predict-world-events/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Thinking Machines Lab and NVIDIA Announce Long-Term Gigawatt-Scale Strategic Partnership

<div class="post-cover compact"><img src="https://imgproxy.citrusreader.com/_xrtlN0yJkhnlFJVBARHP3nlXREapEa-3kfTbORYeUM/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvbnZpZGlhLXBhcnRuZXJzaGlwL2ltYWdlcy9qZW5zZW4taHVhbmctbWlyYS1tdXJhdGktbnZpZGlhLWRlYnMtMzEyNy0yLmpwZw" alt="Cover image for Thinking Machines Lab and NVIDIA Announce Long-Term Gigawatt-Scale Strategic Partnership" class="cover-image" fetchpriority="high" decoding="async" loading="eager" width="2400" height="1795" referrerpolicy="no-referrer"><p class="post-cover-caption">Jensen Huang (NVIDIA) and Mira Murati (Thinking Machines)</p></div><div class="post-content-shell"><article class="content"><p>Thinking Machines Lab and NVIDIA announced today a multi-year strategic partnership to deploy at least one gigawatt of next-generation NVIDIA Vera Rubin systems to support Thinking Machines’ frontier model training and platforms delivering customizable AI at scale. Deployment on NVIDIA’s Vera Rubin platform is targeted for early next year.</p> <p>The partnership also includes an effort to design training and serving systems for NVIDIA architectures and broaden access to frontier AI and open models for enterprises, research institutions, and the scientific community.</p> <p>NVIDIA has also made a significant investment in Thinking Machines Lab to support the company’s long-term growth.</p> <p>“AI is the most powerful knowledge discovery instrument in human history,” said Jensen Huang, founder and CEO of NVIDIA. “Thinking Machines has brought together a world-class team to advance the frontier of AI. We are thrilled to partner with Thinking Machines to realize their exciting vision for the future of AI.”</p> <p>“NVIDIA’s technology is the foundation on which the entire field is built,” said Mira Murati, cofounder and CEO of Thinking Machines. “This partnership accelerates our capacity to build AI that people can shape and make their own, as it shapes human potential in turn.”</p> <p>Building powerful AI systems that are understandable, customizable, and collaborative demands advances in research, design, and infrastructure at scale. This partnership provides that foundation, with the shared aim of ensuring that the most transformative technology of our time expands human capability.</p> <img data-zoomable="true" src="https://imgproxy.citrusreader.com/017sNfzarXEiOePoHvepz6KGeNT-tR5MO0yIgjunbN8/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvbnZpZGlhLXBhcnRuZXJzaGlwL2NvdmVyLnN2Zw" style="display: block; width: 100%; max-width: 800px; height: auto; margin: 0 auto;" loading="lazy" alt referrerpolicy="no-referrer"></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/nvidia-partnership/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Tinker: General Availability and Vision Input

<div class="post-content-shell"><article class="content"><p>Today we are announcing four updates to Tinker:</p> <ul> <li>No more waitlist</li> <li>New reasoning model: Kimi K2 Thinking</li> <li>New inference interface that is compatible with the OpenAI API</li> <li>Vision input support with Qwen3-VL</li> </ul> <h2 id="general-availability">General availability</h2> <p>The waitlist is over! Everybody can use Tinker now; <a href="https://auth.thinkingmachines.ai/sign-up">sign up here</a> to get started. See the <a href="https://thinkingmachines.ai/tinker/">Tinker homepage</a> for available models and pricing, and check out the <a href="https://github.com/thinking-machines-lab/tinker-cookbook">Tinker cookbook</a> for code examples.</p> <h2 id="more-reasoning-with-kimi-k2-thinking">More reasoning with Kimi K2 Thinking</h2> <p>Users can now fine-tune Kimi K2 Thinking on Tinker. With a trillion parameters, Kimi K2 is the largest model in our lineup so far. It is built for long chains of reasoning and tool use.</p> <h2 id="openai-api-compatible-sampling">OpenAI API-compatible sampling</h2> <p>Tinker has a standard function for inference:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2); overflow-x: auto;" tabindex="0" data-language="python"><code data-lang="python" class="language-python"><span class="line"><span style="color:var(--fg2)">prompt </span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)"> types</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">ModelInput</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">from_ints</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:var(--fg2)">tokenizer</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">encode</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-green)">The capital of France is</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,))</span></span> <span class="line"><span style="color:var(--fg2)">params </span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)"> types</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">SamplingParams</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:var(--fg2)">max_tokens</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--code-red)">20</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span><span style="color:var(--fg2)"> temperature</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--code-red)">0.0</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span><span style="color:var(--fg2)"> stop</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">[</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-blue)">\n</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">])</span></span> <span class="line"><span style="color:var(--fg2)">future </span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)"> sampling_client</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">sample</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:var(--fg2)">prompt</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)">prompt</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span><span style="color:var(--fg2)"> sampling_params</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)">params</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">)</span></span></code></pre> <p>With this release, we have added OpenAI API-compatible scaffolding for quickly sampling from a model by specifying a path, even while it’s still training. This also means Tinker can now plug-and-play with any OpenAI API-compatible platform. See more information in our <a href="https://tinker-docs.thinkingmachines.ai/compatible-apis/openai">Tinker documentation</a>.</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2); overflow-x: auto;" tabindex="0" data-language="python"><code data-lang="python" class="language-python"><span class="line"><span style="color:var(--fg2)">response </span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)"> openai_client</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">completions</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">create</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span></span> <span class="line"><span style="color:var(--fg2)"> model</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-green)">tinker://0034d8c9-0a88-52a9-b2b7-bce7cb1e6fef:train:0/sampler_weights/000080</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span></span> <span class="line"><span style="color:var(--fg2)"> prompt</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-green)">The capital of France is</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span></span> <span class="line"><span style="color:var(--fg2)"> max_tokens</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--code-red)">20</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span></span> <span class="line"><span style="color:var(--fg2)"> temperature</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--code-red)">0.0</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span></span> <span class="line"><span style="color:var(--fg2)"> stop</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">[</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-blue)">\n</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">],</span></span> <span class="line"><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">)</span></span></code></pre> <h2 id="vision-input-with-qwen3-vl">Vision input with Qwen3-VL</h2> <p>We’ve added two vision models to Tinker: Qwen3-VL-30B-A3B-Instruct and Qwen3-VL-235B-A22B-Instruct. With these, users can process pictures, screenshots, and diagrams for a variety of applications.</p> <p>To input images, just interleave together an <a href="https://tinker-docs.thinkingmachines.ai/api-reference/types#imagechunk-objects">ImageChunk</a> – consisting of your image, saved as bytes – with text chunks. For example:</p> <pre class="astro-code house" style="background-color:#0000000a;color:var(--fg2); overflow-x: auto;" tabindex="0" data-language="python"><code data-lang="python" class="language-python"><span class="line"><span style="color:var(--fg2)">model_input </span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)"> tinker</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">ModelInput</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:var(--fg2)">chunks</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">[</span></span> <span class="line"><span style="color:var(--fg2)"> tinker</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">types</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">ImageChunk</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:var(--fg2)">data</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)">image_data</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">,</span><span style="color:var(--fg2)"> format</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-green)">png</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">),</span></span> <span class="line"><span style="color:var(--fg2)"> tinker</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">types</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">EncodedTextChunk</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:var(--fg2)">tokens</span><span style="color:color-mix(in srgb, var(--fg2) 70%, transparent)">=</span><span style="color:var(--fg2)">tokenizer</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">.</span><span style="color:var(--fg2)">encode</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">(</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:var(--code-green)">What is this?</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">"</span><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">)),</span></span> <span class="line"><span style="color:color-mix(in srgb, var(--fg2) 85%, white)">])</span></span></code></pre> <p>These vision inputs can be used in a variety of applications out-of-the-box, including SFT and RL finetuning.</p> <p>To demonstrate vision understanding in action, we are sharing <a href="https://github.com/thinking-machines-lab/tinker-cookbook/tree/main/tinker_cookbook/recipes/vlm_classifier">a new cookbook recipe for fine-tuning VLMs as image classifiers</a>. Qwen3-VL-235B-A22B-Instruct obtains reasonable accuracy even given just one example per class; performance improves with more labeled data.</p> <h2 id="training-image-classifiers-with-tinker">Training image classifiers with Tinker</h2> <p>To showcase Tinker’s new vision capabilities, we finetuned <a href="https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct">Qwen3-VL-235B-A22B-Instruct</a> to classify images on four classic datasets:</p> <ul> <li><a href="https://data.caltech.edu/records/mzrjq-6wc02">Caltech 101</a>, a dataset of 101 general object categories.</li> <li><a href="https://www.kaggle.com/datasets/eduardo4jesus/stanford-cars-dataset">Stanford Cars</a>, a dataset of car makes, models, and years.</li> <li><a href="https://www.robots.ox.ac.uk/~vgg/data/flowers/102/">Oxford Flowers</a>, a dataset of flower species.</li> <li><a href="https://www.robots.ox.ac.uk/~vgg/data/pets/">Oxford Pets</a>, a dataset of pet breeds.</li> </ul> <p>Since Qwen3-VL is a language model, we frame classification as text generation: given an image, the model outputs the class name. We compare this approach against a traditional vision baseline of finetuning a vision-only model — DINOv2-base. <a href="https://arxiv.org/pdf/2304.07193">DINOv2</a> is a self-supervised vision transformer that was trained to encode images, and is commonly used as a backbone for pure computer vision tasks. For DINOv2, we add a classification head that predicts a distribution over all N classes. Both models are fine-tuned with LoRA.</p> <p>Labeled image data is scarce for many real-world use cases, so data efficiency is the primary measure we look at. We show the classification accuracy when sweeping across the number of labeled examples per class, starting with just a single one.</p> <figure id="fig:qwen-v-dino"> <img data-zoomable src="https://imgproxy.citrusreader.com/H_qXQbX-ZT9ApE60ddu6KxU6ZX5pzt4GwiMHYwUNqiM/raw:1/aHR0cHM6Ly90aGlua2luZ21hY2hpbmVzLmFpL25ld3MvdGlua2VyLWdlbmVyYWwtYXZhaWxhYmlsaXR5L2ltYWdlcy92bG0tZ3JhcGhzLnBuZw" referrerpolicy="no-referrer"> <figcaption>Comparison of fine-tuned Qwen3-VL-235-A22B and DINOv2 performance on simple image classification tasks.</figcaption> </figure> <p>In the limited-data regime, Qwen3-VL-235-A22B outperforms DINOv2. Not only is it a bigger model, but as a VLM, it also comes with language knowledge out-of-the-box (i.e. what a “golden retriever” or “sunflower” is). This general language-and-vision capability of Qwen3-VL makes it readily available for vision tasks beyond classification.</p> <h2 id="happy-holidays">Happy Holidays</h2> <p>Tinker exists to enable builders and researchers to train and customize state-of-the-art models. As always, we look forward to seeing what you build with Tinker. Happy holidays!</p></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/tinker-general-availability/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

Tinker: Call for Community Projects

<div class="post-content-shell"><article class="content"><p>We launched <a href="https://thinkingmachines.ai/tinker/">Tinker</a> to enable builders and researchers to train models their own way, whether they’re conducting studies or customizing models for new applications. We plan to publish regular roundups of the coolest projects from the Tinker community, and <strong>we invite you to submit what you’ve been Tinkering on to be featured on our blog</strong>.</p> <p>Below are some broad suggestions for what we hope to see from the Tinker featured projects, and some specific research directions we would particularly love to see pursued.</p> <h2 id="guidelines-for-tinker-featured-projects">Guidelines for Tinker Featured Projects</h2> <p>We’re interested in featuring ML research projects, AI-enabled research in other domains, custom models, and other contributions. Some examples:</p> <ul> <li>A reimplementation of a research project or tech report using Tinker, such as papers that compare algorithmic recipes or datasets.</li> <li>Original research in machine learning, such as exploring new approaches to training or optimization or applying novel benchmarks and evaluations.</li> <li>Research in a non-AI field that uses fine-tuned models, such as the work on mathematical theorem provers and chemistry models we <a href="https://thinkingmachines.ai/news/announcing-tinker/#:~:text=Groups%20at%20Princeton%2C%20Stanford">highlighted previously</a>.</li> <li>Product prototypes built with Tinker, demoing a model that does something fresh or delightful.</li> <li>Novel datasets and task environments for training models.</li> <li>High-level libraries built on top of Tinker that enable less experienced practitioners to perform fine-tuning effectively.</li> <li>Infrastructure contributions, such as a clean self-hosted implementation of the Tinker training API.</li> </ul> <p>Your submission should include a write-up and, preferably, an open-source release of your code. We encourage you to focus on rigor and clear evaluation in your write-ups: crisp charts, raw output examples, clear comparisons to alternative approaches or models on relevant benchmarks and metrics. Tinkering is experimenting — we want to feature diligent work and transparent results over novelty or hype.</p> <p>Please send your projects and any related questions to <a href="https://thinkingmachines.ai/cdn-cgi/l/email-protection#4135282f2a2433013529282f2a282f262c202229282f24326f20287e3234232b2422357c072420353433242564737111332e2b242235647371"><span class="__cf_email__" data-cfemail="dbafb2b5b0bea99bafb3b2b5b0b2b5bcb6bab8b3b2b5bea8f5bab2">[email&nbsp;protected]</span></a> with “Featured Project” in the subject line.</p> <h2 id="suggested-research-projects">Suggested research projects</h2> <p>Here are some research directions that we would personally love to see explored and that Tinker can enable real progress on. We have <a href="https://github.com/thinking-machines-lab/tinker-project-ideas">created a repo</a> with detailed motivation and guidelines for each; we’ll be adding more resources to it over time to help researchers get started. We expect most project ideas to surprise us, but this short list could serve as inspiration.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/replicate-cai-with-base-models.md">Replicating Constitutional AI, starting from the base model.</a></strong> Though RLAIF is widely used, it’s most often bootstrapped from existing instruction-tuned models. This makes it difficult to separate the impact of the constitution from the impact of the data-generating model that interprets it. A study of Constitutional AI with and without instruction-tuned models in the pipeline would shed light on the use of constitutions and RLAIF.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/noisy-student.md">RLVR with Noisy student.</a></strong> Noisy student self-distillation was a popular technique in an earlier era of machine learning for making use of large unlabeled datasets, but it hasn’t been adapted widely to LLMs. One possible adaptation is to start RLVR with a small labeled training set and a large unlabeled one, then have the student apply labels to the latter set after each RL run and iterate.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/on-policy-context-distillation.md">On-Policy Context Distillation.</a></strong> Context distillation trains a student model with empty context on a teacher model with long and detailed context. Prior work used off-policy distillation — fine-tuning on teacher samples. We have found that <a href="https://thinkingmachines.ai/blog/on-policy-distillation/">on-policy distillation</a> is often much more effective; it would be useful to compare the two approaches for context distillation in particular.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/memorization-empirical-study.md">RL memory test.</a></strong> Our <a href="https://thinkingmachines.ai/blog/lora/">post on LoRA</a> presented theoretical arguments on the rate of information acquisition by both SFT and RL. You can set up a toy environment where RL must learn a completely random number sequence, to compare the empirical learning rate under various reward functions to the theoretical estimate.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/direct-rl-on-pairwise-judge.md">Direct RL on pairwise judge.</a></strong> RLHF and RLAIF use datasets of pairwise preferences, which are used to train a reward model, which is then used in RL. As an alternative “direct” approach, we can do RL using a prompted model that does pairwise comparisons, without training the reward model. It would be interesting to do experiments comparing the direct and indirect approaches.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/replicate-open-character-training.md">Replicate Open Character Training.</a></strong> Replicate the recent paper on <a href="https://arxiv.org/abs/2511.01689">Open Character Training</a> using Tinker.</p> <p><strong><a href="https://github.com/thinking-machines-lab/tinker-project-ideas/blob/main/gan-joke-generation.md">GAN for jokes.</a></strong> In domains such as humor, it is easier to curate a human-vetted set of demonstrations than to train a reliable judge or reward model. Try implementing GAN-style training for a joke evaluator and joke generator that can craft a joke with a requested subject and keywords.</p> <h2 id="tips-for-high-quality-ml-experiments">Tips for high-quality ML experiments</h2> <p>In closing, we want to offer a few guidelines for running quality ML studies, the same guidelines we strive to adhere to internally when running experiments and documenting the results.</p> <p>We encourage researchers to apply multiple analyses for examining each result. When creating datasets or environments, we recommend training a range of models and applying different evals. When developing novel methods, we suggest comparing to simpler baseline methods and sweeping hyperparameters that performance is sensitive to, particularly learning rate.</p> <p>We’d love to see your reasoning in the write-up: assumptions you made, how your approach diverges from previous reports, and what motivated each change. We hope to see examples of the raw data and model rollouts, along with the summarized results. Finally, we appreciate crisp and detailed write-ups with clean and well-labeled charts and illustrations of the inner workings of the methods used.</p> <p>We are excited to see what our community creates with Tinker, and hope that our featured projects will inspire your own work.</p></article></div><div class="paginator"> <a href="https://thinkingmachines.ai/news/call-for-community-projects/#" id="back-to-top" class="link" title="Go to top">back to top</a> </div>

阅读更多

推荐订阅

Provides a better reading experience (full articles) over the official ones.

Provides a better reading experience (full articles) over the official ones.