Criteria for assessing the effectiveness of distance learning using various models. Recommendations for increasing the effectiveness of distance learning The effectiveness of distance learning in an organization

The innovative idea of ​​distance learning proposed in the project of MBOU Secondary School No. 14 in Khimki requires the development of a system of performance criteria. There are sources in the pedagogical literature that discuss various approaches to assessing this indicator. However, they prioritize the effectiveness of the learning process and do not take into account economic indicators, which play an important role in any project. Therefore, let us approach this issue from an economic position.

We will consider the effectiveness of distance learning as a coefficient equal to the ratio of costs (resources, costs, etc.) to the result obtained and expressed as a percentage. Then, the closer the value of this coefficient is to zero, the more effective the proposed distance learning model is.

In this case, we will evaluate effectiveness using criteria, where the assessment of each has a numerical expression that is as simple and understandable as possible within a certain scale, the same for all criteria.

Let's divide all the criteria into two large groups: quantitative And quality.

Qualitative ones, in turn, will be considered in terms of indicators pedagogical effectiveness for students and indicators pedagogical effectiveness for teachers.

• 
  costs of equipment necessary to implement a particular model of distance learning;
• 
  costs of training personnel to ensure the progress of the educational process in distance learning according to the proposed models;
• 
  costs for wages teaching and other personnel involved in the project;
• 
  total educational time (in hours) allocated for the implementation of a particular model;
• 
  costs of ensuring uninterrupted access to the Internet, with high quality communications;
• 
  time spent by teachers preparing for lessons, preparing and delivering materials and assignments to students.

• 
  the presence of a regulatory framework of documents regulating distance learning for children using various models;
• 
  level of qualifications of teaching staff;
• 
  the complexity of implementing the distance learning model;
• 
  the complexity of constructing a distance learning course according to one model or another.

• 
  level of intellectual development of the student (for example, quality of knowledge, degree of training);
• 
  the number of requests for certain courses within the proposed modules;
• 
  frequency of children’s participation in competitions, shows, competitions, etc.;
• 
  the effectiveness of student participation in various activities;
• 
  results of the final certification of students;
• 
  number of participants in the learning process (volume of the target audience);
• 
  number of people enrolled in universities and other educational institutions.

• 
  student performance indicators;
• 
  motivational stability of the teacher;
• 
  complexity and demand for knowledge in courses;
• 
  development of student performance assessment systems;
• 
  development of a system for monitoring the work of a teacher;
• 
  rationality and efficiency of equipment use;
• 
  student assessment of learning outcomes.

Table 1. Assessment form quantitative cost criteria

Table 3. Assessment form quantitative outcome measures

Table 4. Evaluation form qualitative outcome criteria

Such tables can be complicated by adding the necessary criteria to them, based on the specifics of a particular institution. They are compiled for each learning module with a distance component, introduced into the main educational process. After which the final value of the efficiency coefficient of each model is calculated - formula 1.

- the value of each cost criterion (in points),

- the value of each of the outcome criteria (in points).

In order to achieve high values ​​of this coefficient, the activities of the institution must follow a certain policy that delimits and determines the work of all structures involved in the project. This system of actions includes the following points:

• 
  selection and organization of one or another learning model using distance technologies;
• 
  selection of teachers involved in the project;
• 
  arrangement of training hours in the schedule grid;
• 
  organization of advanced training courses for teaching staff;
• 
  organization of control of the educational process;
• 
  generation of reports on the effectiveness of a particular model;
• 
  formation of an expert commission to assess the effectiveness of the learning process;
• 
  formation of a regulatory framework regulating the process in compliance with laws, codes and other government documents.

• 
  who develops training courses;
• 
  how deadlines for completing work on creating training courses are set;
• 
  what can be included (should not be included) in the training material;
• 
  how the training course should be structured;
• 
  what distance learning methods exist and how to use them;
• 
  how the finished course will be tested.

• 
  there is a proprietary program for automated scheduling of the educational process;
• 
  a form has been developed and logs are kept to monitor the educational process, estimating the number of hours using the equipment, software, laboratory equipment, Internet resources;
• 
  work programs for organizing lessons with a distance component;
• 
  tables for monitoring the efficiency of equipment use;
• 
  materials of pedagogical advice on organizing a distance lesson, types of lessons, filling them with content;
• 
  speeches by colleagues at regional, municipal and school level events to exchange experiences;
• 
  a video library of examples of lessons with a distance component for children with disabilities is being formed.

Our institution plans to solve this problem within the time frame for introducing the project into the life of the school.

The modern method of distance learning (DL) via the Internet has become a logical continuation of earlier methods of obtaining “remote” education. A lot is written about it, it is advertised and called a worthy alternative to “face-to-face” education. However, the question of its effectiveness is practically not covered. And this is very important, since the popularity of distance learning is constantly growing, which means an increasing number of specialists are gaining knowledge in this way.

Modern distance learning method(DO) via the Internet has become a logical continuation of earlier methods of obtaining “remote” education: correspondence, correspondence, television, etc. A lot is written about it, it is advertised and called a worthy alternative to “face-to-face” education. However, the question of its effectiveness is practically not covered. And this is very important, since the popularity of distance learning is constantly growing, which means an increasing number of specialists are gaining new knowledge in this way.

Prerequisites for the growing popularity of distance learning

The main factor determining constant growth popularity of distance learning, is that this method allows one to overcome limitations in the availability of education associated with geographical and spatial factors. At the same time, distance learning courses are often offered today at a more affordable price than “face-to-face” courses. Which, however, is quite logical: organizing the educational process using distance technologies requires significant costs only at the initial stage (which in most universities has already been successfully completed a long time ago), and in the future does not require serious additional financial investments.

In other words, receiving education using distance learning allows you to:

• save money - not only do pre-service courses cost less than full-time ones, but you also don’t have to spend money on accommodation and food in another city;
• independently plan the learning process - the student chooses how and when it is convenient for him to study the educational material;
• receive education without interrupting work - a self-planned schedule of “lectures” allows you to combine work with study.

It is quite natural that students, constrained by time, geographic and financial constraints (and there are more and more of them every year), make their choice in favor of distance learning. But how effective is such training?

Types of distance learning

Initially, even at the dawn of the formation of the BS, it was clear that despite all benefits of distance learning, this method is devoid of those main factors that provide (or can provide) high quality learning in a traditional classroom or individual education situation: the presence of a talented and charismatic teacher who is able to develop a student’s interest in the subject, captivate him, encourage him to comprehend and solve problems, and also maintain high self-esteem and self-confidence.

Obviously, all this is probably missing in teaching by correspondence and - with a very high degree of probability - in television lessons. But this state of affairs was considered a normal cost of the method, a payment for the opportunity to study at a distance and without special expenses. The advent of technology has provided greater possibilities for supplying material and greater ease of use of the method.

Modern distance education is represented by two types:

• synchronous - classes are carried out in real time. That is, at the appointed time, a student (or a group of students) “meets” online with a teacher who gives lectures, answers students’ questions and asks counter questions to check the level of mastery of the material. As a rule, virtual classes are conducted using the Internet and web cameras.
• asynchronous - a classic version of preschool education, in which the student is sent educational material for independent study, followed by passing tests and exams. At the same time, a student can take tests both in person and remotely.

The effectiveness of distance learning

It's not difficult to guess that synchronous distance learning, which differs from the classic “face-to-face” only in the way the teacher interacts with students, is not inferior in effectiveness to its traditional counterpart. True, there is no need to talk about time independence here, since the student must go online at a clearly defined time. And the cost of such training, even if it is lower than that of full-time training, will not be much lower. Therefore, of the three main advantages of distance learning listed above (financial, time and geographic accessibility), only geographic independence is evident.

But the efficiency asynchronous distance learning is very doubtful, since here a lot depends on the student’s ability to learn, the level of his preparation, the correspondence of the complexity of the chosen course to the student’s capabilities, the availability of support and assistance from the teacher, etc. We should also not forget about the student’s motivation.

Experienced teachers believe that “... when the motivation to learn is very low, one can, without risking a mistake, assume that potential academic success will be reduced to some extent.” The specificity of the asynchronous method of DL in the matter of motivation for learning is manifested in the absence of standard motivating moments for traditional learning, such as group pressure, a familiar learning situation, social factors, etc.

In other words, the effectiveness of asynchronous distance learning largely depends on the student himself: on his desire to learn and gain new knowledge, the ability to self-organize and the desire for self-improvement.

Conclusion

Talking about effectiveness of distance learning– and more broadly: education with the help of new information and communication technologies, one can draw an unambiguous conclusion. Yes, it can be considered quite effective, but only if it is a synchronous learning method or the student has strong enough motivation to study the material independently. In other cases, distance learning is more of a self-deception and a way of complacency, allowing you to get only “crusts”, but not the proper level of knowledge.

What to pay attention to! When writing this article, the author tried to present the material as best as possible. in simple language. But, quite possibly, he was not able to do this fully due to the very complex topic of the article.

Electronic distance learning has become firmly established in the corporate training system of Russian companies. And it’s not surprising - this form of training allows you to quickly improve the skills of a large number of workers directly at the workplace, practically without interruption from the production process.

It should be noted that the distance e-learning system as a whole consists of many different electronic courses. These courses are developed, tested for functionality, and then put into production. With the help of electronic courses, bank staff improve their skills. The effectiveness of the entire educational system depends on the effectiveness of each individual training course.

For five years, Bank Vozrozhdenie has been successfully operating a distance e-learning system (hereinafter referred to as SDEL). The expression “functions successfully” refers to the following indicators:

Quantitative: the number of training courses developed and put into operation, the number of employees visiting the “Training and Development” web portal, the number of employees who have completed training in each training course.

Quality: change in the level of knowledge before and after completing the electronic training course, employees’ assessment of the quality of the developed training courses, managers’ assessment of the quality of training of subordinates.

Employees of the bank's training center constantly monitor the above indicators in order to control the quality of the training provided. But these indicators cannot fully reflect the effectiveness of the implemented electronic distance learning system.

How to measure the effectiveness of SDEO?

After the SDEO was implemented at Vozrozhdenie Bank, the first ten training courses were developed, regular staff training began, and the need to evaluate the effectiveness of this form of training was determined.

Accordingly, to key employees Training center The task was set to develop an assessment of the EDS. In 2009, we did not find in the business literature of partners who are engaged in similar activities the necessary samples of assessment of the EDS that would meet the following criteria: the assessment should be as clear as possible, simple, objective, practical and flexible.

Therefore, the staff of the Training Center independently began to develop such an assessment.

1. First, the term “effectiveness of electronic distance learning” was defined.
2. A mathematical model for calculating efficiency has been compiled.
3. Criteria influencing efficiency were identified.
4. Test calculations were carried out.
5. After receiving the first data, the calculation model was slightly modified.
6. After which, the model was put into operation.

IMPORTANT.

This model for calculating the effectiveness of SDEO was developed exclusively for the purposes of the Vozrozhdenie Bank Training Center and is quite possible for other purposes, and may not be applicable to other companies.

Model of effectiveness of SDEO of the Vozrozhdenie Bank Training Center

The term “effectiveness of a distance e-learning system” means the percentage ratio of the results (the totality of resources expended, costs) with the results obtained.

The result of this ratio will be a numerical value between zero and 100%.

Attention! The closer the calculation result is to zero, the higher the efficiency; the closer the calculation result is to 100%, the lower the efficiency.

If the gap is zero (nothing is invested), and the resulting result is maximum, the SDEO turns out to be very effective. Could this be possible?

And vice versa. We invested 100%, but there is no result, how can this be?

Experimentally, by modeling various situations, it was revealed that the most effective EDMS system will be when obtaining calculated data close to 30%. When receiving calculated data close to 80% and above, the SDEO will be practically ineffective.

What criteria are combined into the PROTORI category?

Protori has five criteria. Each criterion is assessed on a ten-point scale.

The higher the score, the higher the rate will be, that is, costs and expenses increase. High protory must be reduced by all means.

The lower the score, the lower the rate will be - costs and expenses will decrease. It is necessary to strive for a low protory.

Let's describe each criterion:

1. Availability of documentation for SDEO

What documents are required to get 10 points?

“Policy for organizing electronic distance learning”.

The more detailed the procedures and requirements for this type of training are, the easier it is to integrate the learning process. Any employee of the company - from an ordinary employee to the head of a department - should be able to find answers to all questions regarding distance learning in this document. Sections that should be included in this policy:

• who is the organizer of the training;
• on whose initiative training courses are assigned;
• how training courses are assigned to employees (time, deadlines, documents);
• where, how, when the employee will undergo training;
• how control occurs;
• how, who, when, to whom reports with results are sent;
• areas of responsibility of employees of different departments in the field of SDEO.

A document that lists all the stages and requirements for the course development process. It is advisable to include the following sections in this policy:

• who can act as a customer for training courses;
• how the customer initiates the procedure for their development;
• who, how, when forms the working group;
• how deadlines for completing work are set;
• who can act as course developers;
• what can be included (should not be included) in the training material;
• how the training course should be structured;
• what distance learning methods exist and how to use them;
• how the finished course will be tested;
• how the course will be put into operation;
• how to interact with external providers (if they are involved).

2.Difficulty in implementing an electronic course

For the simplest implementation, the course receives 1 point. For the most difficult - 10 points. Of course, when creating a training course, you should strive for the simplest implementation, but the way in which training materials are presented is a very important element.

IMPORTANT.

The complexity of technical implementation can be unreasonable or justified. Unreasonable implementation- this is when a very simple material is “poured” in a very technically complex manner. WebSoft's CourseLab software product (Vozrozhdenie Bank employees use only it), which is designed specifically for creating “fill in” educational courses, has all the necessary built-in tools for implementing very interesting courses. Using the built-in CourseLab tools is the right decision from the point of view of “simple implementation”. At the same time, some employees who are engaged in “filling” use program codes, animation and additional tools to add “charm” to the training course, which often makes its implementation very difficult.
Grounded implementation is when complex knowledge needs to be transferred to employees. To convey them, appropriate business cases have been invented that may be impossible to implement using conventional means.

In order to personally evaluate different approaches to the implementation of training courses, you can visit the website courselab.ru section “Competition 2011”. This section of the site contains training courses that took part in the competition for the best electronic course. The training course “Making a Presentation” by Vozrozhdenie Bank took prizes in three categories. This course is very highly interactive: it requires you to complete business cases, solve complex problems, and look for solutions. From the point of view of technical implementation, it is quite simple, because this implementation was carried out using standard CourseLab program tools.

3. Level of knowledge complexity

4. Duration of training course development

At Bank Vozrozhdenie, electronic courses are developed according to a calendar plan approved for one year. The required time is allocated for each course.

If the development time was significantly reduced relative to the planned one, then one point is assigned. If the time is exceeded several times, then ten points.

5. Qualification of specialists - course developers

Let us very briefly describe what roles exist?

Course customer. He sets goals and objectives for the development team; Takes the course into operation.

Course Architect. The course architect is the leader of the entire project. He must determine deadlines, distribute responsibilities, and minimize possible risks. And one more task is creating a script. The script is created jointly with the “Content Developer”. What does the concept of script include? The sequence of presentation of the material, the identification of parallel branches of training, possible branches, when and to what extent to use test questions. At the Training Center of Bank Vozrozhdenie, the course architects are the leading specialists of the Training Center, since they cope with this role better than others.

Content Developer. This specialist must know perfectly the banking product and/or the business process that will be discussed in the training course and/or how the banking program works, and be able to describe all this in an accessible language. Describe in such a way that any user can figure out what to do and how to do it. Content developers are specialists from business units.

Content uploader. The task of this specialist includes the correct design of educational materials, arranging them in the right order, programming the required number of functions and, finally, providing a pre-finished version of the course to begin the procedure for checking it for performance purposes.

Course tester

The process of creating an e-learning course is very similar in structure to creating software. At the end of work on the course, the correctness of its functioning is checked by a specialist in the role of “tester”. He must check its functionality, identify possible errors, inaccuracies... He must pass on the results of his work to the “filler”. To fully test the course, it is advisable to select the most qualified specialists from those departments whose specialists will be trained in the future.

The higher the qualifications of the specialists involved in developing courses, the higher the score - the protory will be higher.

The lower the qualifications of these specialists, the lower the score.

As an example: if the course development is carried out by a student, then the costs will be quite low in this category. You should not hire the most highly qualified programmer to write simple web templates: both you and a college graduate can handle this task.

Let's summarize.

Protori needs to be reduced. Due to what? These, in my opinion, are the most important points that influence the performance improvement indicators of the SDEO. This:

1. Preparation of detailed documentation on the SDEO, the study of which allows for better development of any training courses.
2. Simplification of the technical implementation of the course. Using built-in functions and methods of software for developing training courses.
3. Compliance with and reduction of deadlines for the development of electronic courses.
4. Attracting personnel with the required qualifications. 5. What criteria were combined into the result category?

The higher the score, the higher the result will be.

The lower the score, the lower the result will be.

Let's describe each criterion:

Evaluation of the course by training participants.

Participants' assessment of the training is a very important criterion. At Vozrozhdenie Bank, this procedure was automated immediately after implementation.

1. An electronic questionnaire was created. The WebTutor program from WebSoft, on the basis of which the entire electronic distance learning system in the bank is built, makes it possible to quite simply automate such procedures.
2. The questionnaire was built into the process of completing the training course.
3. Before completing the course, the employee must go to the questionnaire and fill it out - give grades for the training course. All assessments are summed up and the system displays a score that shows how satisfied the training participants are with the training course.

If the course score is 1, then this means very low satisfaction with the training. A score of 10 means very high satisfaction with the training.

Knowledge test results

One of the criteria reflecting the effectiveness of training is the level of residual knowledge, which is measured in the period from one to three months after completion of training. At Vozrozhdenie Bank, knowledge levels are measured using the electronic testing module of the WebTutor program. The higher the level of knowledge acquired, the higher the score for this criterion.

Direct impact of knowledge on business

The list of necessary knowledge that bank employees must possess is quite extensive. This includes knowledge of corporate culture, banking legislation, requirements described in internal documentation, customer service standards and others.

At the same time, some knowledge may have a weak impact on business, while some, on the contrary, may have a very strong impact on business.

As an example, knowledge of sales management of a business unit.

• This knowledge, if possessed by an ordinary employee who is not directly involved in sales management, has little impact on the business.
• And in the hands of a leader, they turn into a powerful tool for achieving results.

Demand for the training course

A high assessment of the demand for a training course is characterized by high interest on the part of the customer, as a rule, these are heads of business units in the development of a training course. In the case when the head of a department is interested in developing a training course, he is allocated the most highly qualified specialist, with additional powers being granted to him, the customer himself personally controls the progress of course development, etc.

If the demand for the training course is as high as possible, the score will be 10; in the opposite situation, the score will be one.

Number of training participants (target audience)

At Bank Vozrozhdenie, employees are assigned courses exclusively at the request of business unit managers. Applications for training are submitted electronically through the Training and Development web portal. After this, training courses are automatically assigned based on received applications.

Since this process is fully automated, it is very easy to control the entire training process, including: the number of training participants, the number of those already undergoing training, the number of those who have completed the training course.

The more training participants, the higher the score. And vice versa.

On the results of the assessment of the effectiveness of the SDEO.

In order to present the actual results of assessing the effectiveness of the EDMS, seven different training courses were selected for this article. They were developed at different times by different employees. Each course was assigned points in the Protories section and in the Results section. As a result, the completed table “Expert model for calculating the SDEO” was obtained.

According to the data obtained, two training courses, course No. 1 and No. 5, have good effectiveness. And courses No. 6 and 7 are extremely low in effectiveness.

If you look at the completed table, it immediately becomes clear that:

1. Course No. 6 is difficult to implement, it involves the presentation of complex tasks, and the results have a “low” impact on business.
2. Course No. 7 is difficult to implement and took a very long time to develop, and the number of participants is small.

Using this approach in assessing the effectiveness of the EDMS, you can:

1. When planning the development of training courses, evaluate their possible effectiveness.
2. During course development, evaluate the resulting effectiveness.
3. Upon completion of training, evaluate real effectiveness.

If you don’t understand the general meaning of such information, then everything is simple: with this information you can understand exactly how you can determine the effectiveness of distance learning.

By the way, this question is asked much more often than others to developers at various presentations of electronic solutions. When answering this question, many immediately begin to refer to the well-known methods of D. Philips, D. Kirkpatrick, Tyler and others. Others explain it in their own way, largely incomprehensible to others.

One may ask another question: “To what extent will the use of such methods be justified? Are there any alternatives to standard methods for assessing the effectiveness of distance learning?” That is, how relevant will the assessment methods used be?

Criteria by which the evaluation method is chosen.

Not so long ago, F. Nichols decided to formulate a list of questions through which it is possible to identify tasks for evaluating training:

Should it be assessed or should it not be done?

For what purpose should the assessment be carried out?

Who or what needs to be assessed?

When to give an assessment?

How should it be assessed?

Do the data needed for the assessment exist and to what extent are they accurate?

How and where exactly will the final grades be used later?

What reserves will the assessment require, and will it still make sense after that?

It was decided to exclude some of the questions from the original questionnaire because they did not concern the target audience, but the personnel undergoing training, while other questions were added taking into account the experience gained.

It is worth starting from the fact that it is the goals of the assessment and the tasks of people that will dictate the direct choice of method. This is a fairly significant statement, because these are the questions that will determine the criteria for these methods. At the same time, it is quite obvious to everyone that a variety of methods will be used in order to find the correct answers to specific questions. Moreover, some of them will provide more information than others.

In a separate order, it should be noted that all questions regarding performance assessment will necessarily be interconnected. The higher quality will be the distance learning that is properly assessed and gives excellent results in the stated aspects. In addition, it is reliably known that estimates in some way distort the true results. That is, on the one hand, it will not be able to track many of the most important indicators, but on the other hand, it will act as a kind of means of guiding the system being assessed, while giving it a certain direction for development.

In order to confirm and explain all this clearly, it is worth considering the following example. Two organizations (they will be called "Master-Telecom" and "FUD") decided to implement a distance education system in order to train personnel involved in the sales department. Naturally, both companies want to know how effectively they were able to invest in training. For this purpose, both companies were able to identify several basic criteria. At Master Telecom, fiscal funds are especially valued, and therefore their project manager calculates how much the organization’s turnover has increased with the introduction of a distance learning system. As a result, the average range of sales per department employee was determined both before and after the start of training. The results turned out to be quite disappointing: the jump in sales was very small, and the project as a whole was absolutely unprofitable.

FUD values ​​capital in a similar way, but they look much further. Therefore, the project manager had to evaluate the impact of the distance education system on the organization as a whole. Their manager, naturally, spent almost a week more on evaluating this than his colleague from the Master Telecom company, however, the results turned out to be different from those previously considered. Despite the fact that sales did not grow that much at the time of assessment, they had a clear trend of stable growth. As it turned out, after the distance learning system was introduced, staff turnover decreased significantly. The head of the HR department found that the staff noted the positive interest of their management and believed that the company cared about them and was no longer selfish. In addition, the HR manager was able to calculate to what extent the organization’s costs for hiring new employees have decreased.

Because the manager had this information, he was able to clearly demonstrate to managers that the distance learning project was very useful for the company.

However, Master Telecom only assessed the growth of its sales...

At FUD, assessment was carried out according to all the most important parameters of existing activities. They focused not on obtaining short-term results, but on sustainable long-term growth. Numerous parameters were assessed, including the number of additional transactions, client satisfaction and the general atmosphere in the team of employees. FUD managers realized that their success would be assessed by many criteria, and their own remuneration would subsequently directly depend on them. All these factors stimulated them to adhere to certain rules of behavior, which determined the strategic success of the organization.

Six months later, at an important conference called "EXPO-Trainings", there was a chance meeting of project managers involved in the implementation of a distance education system in the companies "FUD" and "Master-Telecom". After they started talking, everything eventually came down to discussing their projects. The manager from Master Telecom left the conference very upset, envious of his colleague's success. And the manager from FUD was able to analyze the conversation in detail and compared the approaches that were used in both projects. As a result, he came to the conclusion that by modifying the evaluation parameters it is possible to perfectly manage personnel, while simultaneously stimulating employees to specific behavior.

In the distance learning system, the FUD organization after some time implemented two new training courses on entertaining time management and conflict resolution guidance for dummies. At the same time, in order to be able to test their own hypothesis, their manager decided to spread a rumor that managers would begin to evaluate all managers, including based on the results of the training. Special attention will be paid by managers to the second distance learning program.

After this, it became known that the percentage of completion of training courses was twice as high as usual. Oddly enough, it was the second course that was studied by almost every employee of the company.

From all this we can draw a logical conclusion: absolutely any features in the choice of those means that are aimed at motivating employees are determined by the specifics of the objects on which they influence. In the above example, it was clear that the factors in choosing an assessment method were either the attempt to assess learning, or what part of it was going to be assessed and for what purposes the assessment would be made.

True, there is an additional circumstance that is very often completely ignored: it is necessary to evaluate distance learning as a whole. Since this training is electronic, it becomes possible to use project evaluation methods from the IT field. Indeed, distance education tools are standard software packages that differ little, for example, from Microsoft Office from a conceptual point of view.

However, in assessing distance learning from the point of view of an educational event or as an IT project, there is a considerable difference in both approaches and views. When evaluating an “educational” project, qualitative and quantitative criteria are more in demand. In contrast to these methods, many methods for evaluating IT projects relate to quantitative and financial indicators.

What is used and what can be used

In the West, the following methods are used to evaluate the effectiveness of corporate training:

D. Phillips or D. Kirkpatrick;

Tyler's Target Method;

Naturalistic Guba method;

Applied theory by P. Kearns;

Scrivens method;

Bench marking;

Staphlebeem CIPP Scheme;

CIRO model;

Scheme of B. Aaron V.

Domestic experts, as a rule, are limited to only five names from this list. Most often, the choice of assessment tools is based (with some exceptions) on a superficial way of searching for available information in a variety of open information sources and on the experience of one’s colleagues. In most cases, unfortunately, this can lead to thoughtless copying of different methods and, ultimately, to useless or incorrect assessment.

As already indicated above, in the practice of assessing distance learning, the greatest preference (and perhaps even a whole monopoly) is given to quality assessment methods. Almost all functional branches of companies have already gone through this path: sales departments, personnel management, marketing, production.

The more difficult it is to measure the activities of departments, the more time it takes to find and adapt the tools necessary for assessment.

Despite the fact that distance education differs significantly from other processes in the company in terms of impact, this difference is not that big. Therefore, you should not assume that it is impossible to quantify the results of distance learning.

However, even before introducing examples of such methods, there are certain basic concepts that need to be considered that can justify the difference between the qualitative approach, which is especially popular among T&D specialists, marketers and HR managers, and the quantitative one, which is the most accurate. The latter approach has taken root well among IT specialists and financiers. Quality indicators will greatly simplify the identification of any impacts on a specific object under study, but at the same time, everyone always tries to operate with more vague conclusions that affect not only direct connections, but also correlation. Although identifying quantitative indicators is associated with considerable costs and significant difficulties, it can pay off through higher accuracy of the information identified, coupled with less possible risks incorrect assessment of connections. In most cases, each method represents a certain synthesis between quantitative and qualitative assessments.

Such methods can be divided according to the factors that are measured (indicators or metrics), namely:

Those facing the future tense (forward on the time scale) are predictive;

Showing the immediate result of the impact in the past tense (facing into the past) - past.

Absolutely all effective assessment methods include quality indicators, which serve as a specific forecasting tool, and quantities, which evaluate the results of past time periods. The most developed of all known methods is the system that takes into account balanced indicators.

A system that takes into account balanced indicators and its advantages

There is a lot of information and written works based on the indicated methodology. However, if you exclude a very detailed description, you can determine the most basic factors and the connection with other famous concepts.

This methodology, called the balanced scorecard, offers several areas of assessment that are responsible for all the significant and important issues in the organization and are complexly dependent on each other:

Financial resources;

Clientele;

Entrepreneurial processes;

Development.

All of them, by the way, are closely interconnected and are necessary for any company to function successfully. But for a variety of organizations, each of the designated areas will have different value and weight. For example, an area called development may have much more weight in one firm (consulting) than in another (fast food restaurant).

For any of these areas, quantity indicators are selected, with the help of which you can determine the effectiveness of work in any of the areas. It is worth pointing out several very significant distinctive features of a system that takes into account balanced indicators:

Flexibility of concentrated indicators.

This factor will be expressed in regular reassessment of goals and a direct set of indicators for evaluation. In this case, preference will be given to those that will be more relevant to the organization, that is, its mission and strategic approach. This will allow the assessment system to simultaneously evolve together with the object that will be assessed. In other words, the system that takes into account balanced indicators is the only one of all the studied methods that is not subject to obsolescence and can be modified along with the organization;

The concept of managing only what can be measured.

Methods for assessing effectiveness do not include assessing the number of indicators of a specific activity. There are sometimes certain exceptions, where it can be noted that quantity indicators are created for many quality indicators. But only in a methodology that takes into account balanced indicators as one of the dominant and fundamental ideas does a concept emerge, taking into account which it is possible to manage only what is generally possible to measure. Such a doctrine will mean that absolutely all known indicators will be subject to measurement of quantity, and for absolutely every direction it is possible to select the desired indicator.

Such features very favorably distinguish the system, which takes into account balanced indicators from other assessment methods, while using this assessment system for management purposes. At the same time, it is possible to very significantly increase the effectiveness of the technique from an economic point of view;

The appearance of additional indicators.

Many traditional fiscal indicators imply very rigid sets of values ​​that need to be assessed, as well as the identification of important priorities for the development of the company, while denying the use of other indicators in the assessment. A large percentage of managers, based on their own experience, use several sets of such indicators. The next most important advantage of this system, which takes into account balanced indicators and its advantages, is the ability to include ready-made soft or, on the contrary, hard indicators in the entire assessment scheme. For example, financial indicators may include the total cost of ownership, additional economic price, level of return on investment, and the like. At the same time, in the direction that implies processes, clientele and development, one can completely include the method of D. Kirkpatrick, the scheme of B. Aaron and almost every quality methodology for assessing the effectiveness of distance learning (except for the Guba approach, because it is included in the designated the industry simply doesn’t make any sense);

Increased transparency of all processes.

Another important advantage of the system, which takes into account balanced indicators and its advantages, is the so-called cascading principle used in it. It is he who is able to create a hierarchical chain of tasks and various goals that unite absolutely all activities within the organization into a whole set of interconnected processes. Such a view of the activities in the company will allow you to understand and imagine much more clearly the whole complex of operations taking place within, as well as the contribution of each of the organizational units or functional responsibilities to the final achievement of all final goals.

Important techniques and indicators

As was already noted a little above, a system that takes into account balanced indicators is a complex system in which standard and new various indicators, and often even entire large-scale methods, act as component modules. We can give several examples of how exactly you can select the necessary indicators for the system under consideration:

D. Kirkpatrick's method. This model includes several levels of assessment: acquisition, response, outcome and behavior. Each of them, of course, has its own purpose and its own direction in the system.

Reaction. The standard indicator of quantity is the assessment sheets that are issued to each of the participants. This method of collecting information data directly is often criticized due to the fact that they believe that there is insufficient and poor-quality objectivity, the data is distorted, and all this happens because the estimates are inflated. As an alternative, we can propose the use of indicators of a different type, not direct. By the way, to many this approach seems completely justified.

Assimilation. The quantity indicator may be the result of standardized tests and other verification or validation methods. This level of assessment can provide very important information on the learning process, the development of all students, the quality of work performed and educational requirements.

Behavior. Most often, the designated direction is assessed qualitatively through surveys of management, colleagues and other entities who are in contact with a particular employee.

Bottom line. The result may be a radical change in all work processes; for example, the latest promotion techniques may lead to the growth of a certain part of the market. The system will be able to influence the interaction with both internal and external clients (decrease in complaints, increase in satisfaction), improvement of personal production indicators (increase in its completed sales). That is, for this level of assessment it would be quite logical to include several indicators, and not just one single one.

Total cost of ownership.

This method is aimed at clearly identifying the costs of various information systems at each stage of this system. With the proposed methodology, it is possible to very clearly understand all the schemes, principles and cost structures of such technologies. Costs will be divided into direct and indirect.

Direct costs are carried out through an accountant (salaries, purchases of equipment, software). Indirect costs are more difficult to identify. These will need to include troubleshooting computer problems, travel expenses, distance learning costs, and so on.

As you can see, there are quite a lot of methods and indicators, there is plenty to choose from. Perhaps, after a short time, you will be able to create completely new indicators for your company personally. Don't worry: very often the newest indicators will be the most useful. Do not forget also about the importance of distance education; no matter what indicators you include in your project plan, take this factor into account.

One of the features of the modern pedagogical process is the widespread use of various information technologies. The personal computer and the Internet have become integral components of education. The use of telecommunications and network technologies makes it possible to conduct training in cases where the teacher and student are separated by a significant geographical distance. One of the main modern information technologies in education, which appeared in lately- distance learning. Currently, distance learning is being widely introduced into the educational process and it becomes relevant to determine its effectiveness and the degree of practical use.

Based on a survey of students in general education institutions and their parents from various regions of Russia, we can conclude that the majority of children (64.17%) take part in distance education (Figure 1).

At the same time, children are studying remotely in preparation for passing the EGE and GIA (27.13%), receiving additional education (24.65%), preparing for admission to secondary secondary education and higher education institutions (20.01%), as part of scientific research work ( 14.99%), preparation and participation in Olympiads (7.77%), obtaining general education (4.58%).

Figure 1 – Participation of schoolchildren in distance education

However, the majority of respondents (50.01%) rarely use distance technologies, only 23.8% constantly use distance learning, 22.91% - as needed (Figure 2).

Figure 2 – Frequency of use of distance learning

Figure 3 shows a graph characterizing the most common forms of remote work of a teacher (teacher) with children:

Consulting on educational topics (26,88%);

Transferring the task and receiving the result of the completed task (21.68%);

Testing knowledge through testing (17.73%);

Organization of work on the project (14.79%);

Conducting full lessons (10.46%);

Discussion of educational material (4.51%).

Figure 3 – Forms of remote work of a teacher (teacher) with students

Figure 4 shows a graph characterizing the most common tools used by a teacher (instructor) in the process of distance learning. After analyzing the graph, it can be determined that in the process of distance learning, 17.46% of children, teachers (teachers) use email; 16.79% – Internet chats, 16.75% – e-school, 15.41% – Skype, 11.85% – school website, 10.47% – Internet forum, 8.71% – distance learning system.

Figure 4 – Tools used by teachers in the process of distance learning

Figure 5 shows a graph characterizing the most common means that a teacher (teacher) uses to involve parents in the educational process. 40.3% of parents indicated that the teacher (teacher) uses an electronic diary to involve them in the educational process; 28.56% – email, 26.35% – Skype.

Figure 5 – Answers to the question “What means does the teacher (teacher) use to involve you in the educational process?”

When assessing the effectiveness of using video lessons in the educational process, the majority of parents and students (91.54%) responded that the use of video lessons in the educational process is effective within individual academic subjects (Figure 6).

Figure 6 – Efficiency of using video lessons in the educational process

Figure 7 shows a graph showing what problems, according to students and their parents, distance learning solves:

Figure 7 – Problems solved by distance learning

Distance learning primarily provides the opportunity to study at a convenient time and place (16.16%);

Distance learning develops the child’s abilities (16.10%);

Distance learning provides an opportunity to expand your circle of contacts (15.02%);

Distance learning develops skills in working with volumes of information, develops independence in searching and using the necessary information (14.86%);

Distance learning increases the child’s level of knowledge, skills and abilities (13.88%);

Distance learning creates motivation for self-education (11.79%);

Distance learning develops self-control skills (10.58%).

Thus, distance education solves a wide range of problems, but it cannot completely replace the traditional form of education: the majority of students and parents (46.61%) believe that distance learning serves good tool, complementing classes at school, and 24, 98% - that distance learning cannot replace classes at school. A quarter of respondents (27.34%) still consider distance education as a full-fledged alternative to distance learning (Figure 8).

Figure 8 – Opportunities for distance learning to become an alternative to school classes

At the same time, 67.75% of parents and students indicate the effectiveness of using distance technologies within the framework of additional education for students, and only 30.48% indicate the effectiveness within the framework of basic education (Figure 9).

Figure 9 – Efficiency of using distance technologies within basic and additional education

When determining the main elements of the educational process implemented using distance technologies, 32.45% of students and parents indicated that the educational institution, during the implementation of the educational process, provides students with access to electronic educational resources. 31.62% - organizes the participation of schoolchildren in olympiads, competitions, scientific projects and research, 31.21% - uses video lessons on individual subjects (Figure 10).

Figure 10 – Main elements of the educational process implemented using distance technologies

Figure 11 shows that the majority (72.37%) of students and parents believe that current control of knowledge and intermediate certification is best carried out by combining traditional methods and a computer testing system, 24.48% believe that current control of knowledge is better carried out using traditional assessment methods knowledge.

Figure 11 – Conducting ongoing knowledge monitoring and intermediate certification

The effectiveness of distance learning, according to parents and students, depends on the qualifications of teachers developing video lessons (28.8%), on the qualifications of teachers using educational activities video lessons and electronic educational resources (16.79%), the quality of the video lessons and electronic educational resources themselves (18.68%), the technical capabilities of the computer and the quality of the Internet connection (14.1%). To a lesser extent, performance depends on the child’s motivation for educational activities (12.33%) and on the capabilities of the child himself (8.5%) (Figure 12).

Figure 12 – Factors influencing the effectiveness of distance learning

25.17% of respondents believe that the most effective tool for distance learning is electronic educational resources (EER), 24.98% - real-time teacher consultations (on-line), 22.63% - written teacher consultations upon prior request (off -line), 12.37% – classes under the guidance of teachers in virtual or remote laboratories, 10.59% – video lessons, 1.63% – knowledge control system (Figure 13).

Figure 13 – Effective distance learning tools

At the same time, it is effective to conduct distance learning in the following academic subjects: mathematics, geometry, computer science (33.73%); geography, social studies, life safety (31.29%); history, Russian language, literature, foreign language(27%). The effectiveness of the distance learning process in the natural sciences (biology, chemistry, physics) is very low - 4.62% (Figure 14).

Figure 14 – Academic subjects for which distance learning is effective

Distance learning is effective in teaching children with limited opportunities health (27.09%); for teaching children who missed classes due to illness (19.34%), for teaching children who do not want to attend secondary school (15.01%), in the process of identifying and developing the abilities of gifted children (14.25%) There are parents and students ( 2.93%), who believe that distance learning is not effective (Figure 15).

Figure 15 – Answers to the question “In your opinion, is distance learning effective?”

Having analyzed the data obtained, we can judge the fairly high effectiveness of distance learning in general, as well as the effectiveness of the methods, methods, types, tools, directions and models of distance learning used.