![]() ![]() To gain a general sense of how far nowadays the deep models are from reasoning, the following graph from the GPT-3 paper ( Brown et al., 2020) tells something. For knowing about the concern, it is recommended to read the book of why or the Chinese Room Experiment. This reflects an important weakness of the trained model - the lack of reasoning, which has raised much concern in the deep learning community over the last few years. However, for situations where the code required for generation is unusual such as changing n to another name, it is easy to fool the model to generate some unexpected codes(will talk more about this in Section 4). For example, in Figure 1, the factorial code snippet is generated correctly basically because the model learns from the training codes that most programmers have written it in this way. It is likely to train a deep model big enough to memorize or summarize rules or patterns of statistically commonly-used codes by the real programmers. Facing this bottleneck, now it makes sense to think about if the CCT can be advanced by taking a further step, namely, taking longer context into account for generating useable code in more complicated situations.īased on the current progress of deep learning, the answer is yes but in a semi-automatic way. When there is a bottleneck, people try hard to escape from it although it is usually hard. That says they only perform well in completing short sequence of codes in situations where the generated codes do not so heavily depend on the context of long sequence, such as, in P圜harm, completing a method’s name, variable’s name, etc. Currently, many IDEs are able to auto-complete code but in a limited way. The outline of this blog is organized as follows.Īutocoder is designed for the code completion task (CCT) where a sequence of codes written by the programmer □□ are detected as the context to prompt the automatic generation of the uncompleted codes by a program □. Finally, a list of future pointers to Autocoder will be presented. I will also give some of my personal reflections on the generated codes by Autocoder. This blog first gives an introduction to the project’s background, and then reveals the details of how the dataset is prepared and the fine-tuning process is conducted in programming with Python. ![]() The workflow of Autocoder at training and inference time ![]()
0 Comments
Leave a Reply. |