---

%PDF-1.5 %���� 1 0 obj << /Metadata 3 0 R /Names 4 0 R /OpenAction 5 0 R /Outlines 6 0 R /PageMode /UseOutlines /Pages 7 0 R /Type /Catalog >> endobj 2 0 obj << /Author (Niklas Muennighoff; Alexander M. Rush; Boaz Barak; Teven Le Scao; Aleksandra Piktus; Nouamane Tazi; Sampo Pyysalo; Thomas Wolf; Colin Raffel) /Creator (arXiv GenPDF \(tex2pdf:\)) /DOI (https://doi.org/10.48550/arXiv.2305.16264) /License (http://arxiv.org/licenses/nonexclusive-distrib/1.0/) /PTEX.Fullbanner (This is pdfTeX, Version 3.141592653-2.6-1.40.25 \(TeX Live 2023\) kpathsea version 6.3.5) /Producer (pikepdf 8.15.1) /Title (Scaling Data-Constrained Language Models) /Trapped /False /arXivID (https://arxiv.org/abs/2305.16264v5) >> endobj 3 0 obj << /Subtype /XML /Type /Metadata /Length 1832 >> stream endstream endobj 4 0 obj << /Dests 8 0 R >> endobj 5 0 obj << /D [ 9 0 R /Fit ] /S /GoTo >> endobj 6 0 obj << /Count 10 /First 10 0 R /Last 11 0 R /Type /Outlines >> endobj 7 0 obj << /Count 50 /Kids [ 12 0 R 13 0 R ] /Type /Pages >> endobj 8 0 obj << /Kids [ 14 0 R 15 0 R ] /Limits [ (Doc-Start) (table.caption.36) ] >> endobj 9 0 obj << /Annots [ 16 0 R 17 0 R 18 0 R 19 0 R 20 0 R 21 0 R ] /Contents [ 22 0 R 23 0 R ] /MediaBox [ 0 0 612 792 ] /Parent 24 0 R /Resources 25 0 R /Type /Page >> endobj 10 0 obj << /A 26 0 R /Next 27 0 R /Parent 6 0 R /Title 28 0 R >> endobj 11 0 obj << /A 29 0 R /Count -24 /First 30 0 R /Last 31 0 R /Parent 6 0 R /Prev 32 0 R /Title 33 0 R >> endobj 12 0 obj << /Count 36 /Kids [ 24 0 R 34 0 R 35 0 R 36 0 R 37 0 R 38 0 R ] /Parent 7 0 R /Type /Pages >> endobj 13 0 obj << /Count 14 /Kids [ 39 0 R 40 0 R 41 0 R ] /Parent 7 0 R /Type /Pages >> endobj 14 0 obj << /Kids [ 42 0 R 43 0 R 44 0 R 45 0 R 46 0 R 47 0 R ] /Limits [ (Doc-Start) (figure.caption.48) ] >> endobj 15 0 obj << /Kids [ 48 0 R 49 0 R 50 0 R ] /Limits [ (figure.caption.49) (table.caption.36) ] >> endobj 16 0 obj << /A << /S /URI /Type /Action /URI (mailto:n.muennighoff@gmail.com) >> /Border [ 0 0 0 ] /C [ 0 1 1 ] /H /I /Rect [ 242.003 568.008 364.265 580.91 ] /Subtype /Link /Type /Annot >> endobj 17 0 obj << /A << /S /URI /Type /Action /URI (https://github.com/huggingface/datablations) >> /Border [ 0 0 0 ] /C [ 0 1 1 ] /H /I /Rect [ 230.938 352.594 457.782 363.717 ] /Subtype /Link /Type /Annot >> endobj 18 0 obj << /A << /D (section.6) /S /GoTo >> /Border [ 0 0 0 ] /C [ 1 0 0 ] /H /I /Rect [ 281.241 112.705 293.127 123.609 ] /Subtype /Link /Type /Annot >> endobj 19 0 obj << /A << /D (cite.hoffmann2022training) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 321.539 93.044 333.494 101.791 ] /Subtype /Link /Type /Annot >> endobj 20 0 obj << /A << /D (section.5) /S /GoTo >> /Border [ 0 0 0 ] /C [ 1 0 0 ] /H /I /Rect [ 487.28 89.771 499.206 101.791 ] /Subtype /Link /Type /Annot >> endobj 21 0 obj << /A << /S /URI /URI (https://arxiv.org/abs/2305.16264v5) >> /BS << /W 0 >> /NM (fitz-L0) /Rect [ 12 223.78998 32 568.21 ] /Subtype /Link >> endobj 22 0 obj << /Filter /FlateDecode /Length 141 >> stream x�E�� �0D�����NvӂxD�f�M<��C���� 23�f���N��s��� }ѳ��r�Th�n%���Q�;��t�Z�~]��4Ek���pP��¸�8qz2DH���e�q��jynStuL;�$���', endstream endobj 23 0 obj << /Filter /FlateDecode /Length 2291 >> stream xڝXIs����W�T��Y���b�J��]��W���0�$JXh,��_���
�ԓ��a���_��{[�{�^��ۅ��{�<���TD^^]��bO�,���x��?/��^\^�p:aRD�w��D�X*/NR�Eʻ]{_��\�E� V2��/�׫wM���.j���k]o�5���Y�����~���D�8�X�L_�ro �{�e ۃ=UY9�Wz733��2����1Б���I���/u�������lP��+ �,Q)^O�-�1��tj�DI&�[͎�\�G]�M�0?����%i�"��6o�D���Vߟ!I�jNZ���m�I�"����v�sN/7g�Ŋ)!_�d�2�Dz+�Y'���;0Z����}?t��%�J—���:}j]�ک`5�O�95 ��nt�o���C�˳F�����u�owM5���$��\�bΤ��]��"Uo6�

---