URL: https://proceedings.neurips.cc/paper_files/paper/2014/file/5a18e133cbf9f257297f410bb7eca942-Paper.pdf
%PDF-1.3
1 0 obj
<<
/Kids [ 4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R ]
/Type /Pages
/Count 9
>>
endobj
2 0 obj
<<
/Subject (Neural Information Processing Systems http\072\057\057nips\056cc\057)
/Publisher (Curran Associates\054 Inc\056)
/Language (en\055US)
/Created (2014)
/EventType (Oral)
/Description-Abstract (Deep Neural Networks \050DNNs\051 are powerful models that have achieved excellent performance on difficult learning tasks\056 Although DNNs work well whenever large labeled training sets are available\054 they cannot be used to map sequences to sequences\056 In this paper\054 we present a general end\055to\055end approach to sequence learning that makes minimal assumptions on the sequence structure\056 Our method uses a multilayered Long Short\055Term Memory \050LSTM\051 to map the input sequence to a vector of a fixed dimensionality\054 and then another deep LSTM to decode the target sequence from the vector\056 Our main result is that on an English to French translation task from the WMT\05514 dataset\054 the translations produced by the LSTM achieve a BLEU score of 34\0568 on the entire test set\054 where the LSTM\047s BLEU score was penalized on out\055of\055vocabulary words\056 Additionally\054 the LSTM did not have difficulty on long sentences\056 For comparison\054 a phrase\055based SMT system achieves a BLEU score of 33\0563 on the same dataset\056 When we used the LSTM to rerank the 1000 hypotheses produced by the aforementioned SMT system\054 its BLEU score increases to 36\0565\054 which is close to the previous state of the art\056 The LSTM also learned sensible phrase and sentence representations that are sensitive to word order and are relatively invariant to the active and the passive voice\056 Finally\054 we found that reversing the order of the words in all source sentences \050but not target sentences\051 improved the LSTM\047s performance markedly\054 because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier\056)
/Producer (PyPDF2)
/Title (Sequence to Sequence Learning with Neural Networks)
/Date (2014)
/ModDate (D\07220151202123742\05508\04700\047)
/Published (2014)
/Type (Conference Proceedings)
/firstpage (3104)
/Book (Advances in Neural Information Processing Systems 27)
/Description (Paper accepted and presented at the Neural Information Processing Systems Conference \050http\072\057\057nips\056cc\057\051)
/Editors (Z\056 Ghahramani and M\056 Welling and C\056 Cortes and N\056D\056 Lawrence and K\056Q\056 Weinberger)
/Author (Ilya Sutskever\054 Oriol Vinyals\054 Quoc V\056 Le)
/lastpage (3112)
>>
endobj
3 0 obj
<<
/Type /Catalog
/Pages 1 0 R
>>
endobj
4 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 13 0 R
/Resources <<
/ExtGState 14 0 R
/ProcSet [ /PDF /Text ]
/Font 16 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
5 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 31 0 R
/Resources <<
/ExtGState 32 0 R
/ProcSet [ /PDF /Text ]
/Font 33 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
6 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 34 0 R
/Resources <<
/ExtGState 35 0 R
/ProcSet [ /PDF /Text ]
/Font 36 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
7 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 67 0 R
/Resources <<
/ExtGState 68 0 R
/ProcSet [ /PDF /Text ]
/Font 69 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
8 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 70 0 R
/Resources <<
/ExtGState 71 0 R
/ProcSet [ /PDF /Text ]
/Font 72 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
9 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 77 0 R
/Resources <<
/ExtGState 78 0 R
/ProcSet [ /PDF /Text ]
/Font 79 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
10 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 86 0 R
/Resources <<
/ExtGState 87 0 R
/ProcSet [ /PDF /Text ]
/Font 88 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
11 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 93 0 R
/Resources <<
/ExtGState 94 0 R
/ProcSet [ /PDF /Text ]
/Font 95 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
12 0 obj
<<
/Parent 1 0 R
/Rotate 0
/Contents 96 0 R
/Resources <<
/ExtGState 97 0 R
/ProcSet [ /PDF /Text ]
/Font 98 0 R
>>
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
13 0 obj
<<
/Length 6747
/Filter /FlateDecode
>>
stream
x��=�r�q�
_��ǠյW�,�e��Q�DIi$�i����띙Uݝ�d��c�hVג�Z���<���+����W�'&M1�~>�Ow��8��yN�5�oN�l�?_�� �\���Q4E���7��y��S&��;}��I^D���&���i�a�'oN�v���y��$�ή�aGΥ����Cn����ںaą�i
F�ݽ��^�u��spʫȿ|
�Z��هs�&����{��J��U���>t�ş��ωq�R��B�)��O�\�>Ð�h�S�i�^���i��l�?��n_�9�9Hs�}>������[~�u�+��-�a�V���''?��� �h�c'�[�D��W��V��4Ц9u�O.�L_���0���kړ�ɜ�Ö`l� �M���"���$H��Ma}p�B�Y5�<�0�ݐb<�D�\t,kaeM`ɏx�_�÷o'�����Re���:�2���)��H۾�ٹ�V��|��0��k9S�zIhK:�j}D)?;��4�����������������f@����8m�ϛ�M8{��͇���*�]�nq�[��_$�e��|���p��!���܀s�Vco6ڨ��.���z�1�!�
�M0�pO���>�k#�1�Ӥ�np�i�0#������s�2t��2ȫ���2���G�?�f�Vv��)�:��0k���n����0�=5���*�Vj�5NQ�1Z�d=_�=^o����v�_�F���s���78}�-��G��2��[�p��[6���v��v��oַ�`�9EGP�>�^X�t�_��pjF�{F|G�E�ԉ��@�>c_�Fg^Tf��8]���!�B�����������ύ�L��ٻs��f�E[�c?n��!�x[F��5Na�@������
�F�\�\$��_m�o��������v�6�dk�2���s�EO���U�m�]����� ��1d����̞�V��LBY��s_n��\*. �r(�h$w��.�Ҁ�h��K���}>1*�|OGz~���,�X%DA/6�~Y�k�u~��s�'�|��_��Z�����/2L���b�V�:��
�����0�FM��wCh`ǣ@���7���d�6P^=�
�����?���������)��C��+G�� ��㚟�f�
��:OM2k��9����5�"3��$G5Xm��A� ��oh}|Q���1��(�\(<�$=GY$e��7�y���۬�^��`������w7�-�;~�OZn(G��4�&��}hUF5��`&��ճ��g���'���4�W�p%,�,X�D�z2�U�����8�9Og��9�Ӄ�G8�{�y_wOy6fB���`����$�I4F��`���%��c ���eD�۳!��62M6-l�v~����m���� �-��;p:~�Q��C,1��f�Zj��|���������E���;��g
r����}�`�$jxY3fJ�!S[����@�hϠȊ�6_�\e��,2�cX����8�lC-dF��ʗ��E��5 V��`��M�D7�#5O�n���5���sڬ��C1~�A�!��1LQ��O��������R�:@4T��� r$��j�� ��{��/�:�6h-�1���6#�T!2ö��Ŵ�!�kY_ЈمGe0q�
U��]|(��P�/��q8���ф��f�KSI���^b��!:��ʰ���¼w��bV�L\6����V
�v܄y-㵊DO�#��:��8�1�ި �ݷ8�l�FK�4�>�؎�.,�u˚8��L�TH�B#��O�}3�9I��ɫ�_g�?��B�%�P��a>������g����6�d{��C����Lg�`~'�#���Ufpo��V��ƌ������B��&tF�d�4@����Σ�*#>�&��Cc� �������@�p���0Y�D)�?fxK6.�K�Z+�8���slDo���U9�R�\�,R�*�A��6���!��� Ę��o�k�""wz��,D^q}5w��BQ�C�e�:���a���̖��l�� 3"=� WvIrh)}������1&��D�,��;�#m4n��rmL���>�c����~-��\��%?��ڸRp��V>�x���b'��H|5�����G������s
�-��VB��F�,��`/&������e�އ&���"AHi�6���VpK�F�����}/.�������jm�����s6ߑ����2�_�@v��Uғ��2_���]�l���N���Q�i���Tؑ��k;{4���z�7;�"����B����0�T�t~���}��c�E�
���#��~%Dg�m��7'��B0�#���M��
}Px`��?����We�d��8T�l�ØT�|U�hM*|�ᔼ>NL�<6�6Ao�m��Q��H6�}c> �cIX�bE��'��V��UxK����j9�������b�c�bU�h���H������=
1>h�<����=�Y�4�rt�L-��6Ё���"��}���!�5���ɞ����P���^��8&�����Z���s�5X��g8T'��`(�l@��p���.A�E� c���ab�i���H���=��٩ V�����g,1�c�2���r#�o���]_���9`�"��j�爄7����$�wr�D��H(֧,KU^^�&>
��`��(�^v�������^�bI��F*�*@�F���(�!e� r��P�us~=����@ee۴�J�9�� ^�X&p�[�_φ��gX���� N�L
<�h�u\�~�j���0�"�J�B�e��m���1��\�Ʉ������%�F� v��
.ck��t
�4f�Gn��