rxjs - 如何使用RXJS在多个消费者之间共享资源池

Question

我们如何在 RXJS 中将消费者的工作分配给一组有限的资源？

我这里有一个Pool类(简化版):

class Pool<TResource> {

  private readonly resource$: Observable<TResource>;

  constructor(resource$: Observable<TResource>) {
    this.resource$ = resource$.pipe(
      // We use share replay here, so multiple calls to `schedule` will share the resources
      shareReplay()
    );
  }

  /**
   * Schedules a task to be executed on resources in the pool. Each input is paired with a resource, which allows async work to be done.
   * @param input$ The inputs to pair up with a resource.
   * @param task The task to execute on each resource
   */
  public schedule<TIn, TOut>(input$: Observable<TIn>, task: (resource: TResource, input: TIn) => Promise<TOut> | TOut): Observable<TOut> {
    const recycleBin = new Subject<TResource>();
    const resource$ = merge(recycleBin, this.resource$);

    return zip(resource$, input$).pipe(
      mergeMap(async ([resource, input]) => {
        const output = await task(resource, input);
        //  Recycles a resource so its re-emitted from the `resource$` observable.
        recycleBin.next(resource);
        return output;
      }),
      tap({ complete: () => recycleBin.complete() })
    );
  }
}

你可以这样使用它:

class CalculatorResource {
  expensiveCalculation(n: number) {
    return new Promise<number>(res => setTimeout(() => res(n*2), 1000));
  }
}

const pool = new Pool(of(new CalculatorResource(), new CalculatorResource()));
const input$ = of(1, 2, 3, 4);
const output$ = pool.schedule(input$, (calc, n) => calc.expensiveCalculation(n));
output$.subscribe(console.log)
// ...wait 1 sec
// Logs 2
// Logs 4
// ...wait 1 sec
// Logs 6
// Logs 8

这按预期工作。

但是，当我们并行调用schedule时，资源也会并行分配。这不好，我们希望资源均匀分布，因为它们所执行的任务的性质决定了它们不能并行调用。

const pool = new Pool(of(new CalculatorResource(), new CalculatorResource()));
const input$ = of(1, 2, 3, 4);
const parallelInput$ = of(5, 6, 7, 8);
pool.schedule(input$, (calc, n) =>
  calc.expensiveCalculation(n)
).subscribe(console.log);
pool.schedule(parallelInput$, (calc, n) =>
  calc.expensiveCalculation(n)
).subscribe(console.log);
// Actual output:

// ...wait 1 sec
// Logs 2
// Logs 4
// Logs 10
// Logs 12
// ...wait 1 sec
// Logs 6
// Logs 8
// Logs 14
// Logs 16

// What i would like to see:
// ...wait 1 sec
// Logs 2
// Logs 4
// ...wait 1 sec
// Logs 10
// Logs 12
// ...wait 1 sec
// Logs 6
// Logs 8
// ...wait 1 sec
// Logs 14
// Logs 16

Answer 1

所以最主要的是您需要共享执行工作的实际部分，而不仅仅是资源。

这是我的解决方案:

https://stackblitz.com/edit/rxjs-yyxjh2?devToolsHeight=100&file=index.ts

import { merge, Observable, Observer, of, Subject, zip } from 'rxjs';
import { ignoreElements, concatMap, switchMap } from 'rxjs/operators';

class Pool<TResource> {
  private readonly resourceFree$ = new Subject<TResource>();
  private readonly dispatcher$ = new Subject<{
    execute: (resource: TResource) => any;
    observer: Observer<any>;
  }>();
  private freeResources$ = merge(this.resource$, this.resourceFree$);
  readonly doWork$ = zip(this.freeResources$, this.dispatcher$).pipe(
    switchMap(async ([resource, work]) => {
      try {
        const result = await work.execute(resource);
        work.observer.next(result);
        work.observer.complete();
      } catch (err) {
        work.observer.error(err);
      }
      this.resourceFree$.next(resource);
    }),
    ignoreElements()
  );

  constructor(private resource$: Observable<TResource>) {}

  public schedule<TIn, TOut>(
    input$: Observable<TIn>,
    task: (resource: TResource, input: TIn) => Promise<TOut> | TOut
  ): Observable<TOut> {
    return input$.pipe(
      //you can use mergeMap here as well, depends on how fast you want to consume inputs
      concatMap((input) => {
        const work = {
          execute: (r) => task(r, input),
          observer: new Subject<TOut>(),
        };
        this.dispatcher$.next(work);
        return work.observer;
      })
    );
  }
}

class CalculatorResource {
  expensiveCalculation(n: number) {
    return new Promise<number>((res) => setTimeout(() => res(n * 2), 1000));
  }
}

const pool = new Pool(of(new CalculatorResource(), new CalculatorResource()));
pool.doWork$.subscribe(); //this is to start the pool dispatcher

const input$ = of(1, 2, 3, 4);
const parallelInput$ = of(5, 6, 7, 8);
pool
  .schedule(input$, (calc, n) => calc.expensiveCalculation(n))
  .subscribe(console.log, undefined, () => console.log('1st done'));
pool
  .schedule(parallelInput$, (calc, n) => calc.expensiveCalculation(n))
  .subscribe(console.log, undefined, () => console.log('2nd done'));

setTimeout(() => {
  pool
    .schedule(parallelInput$, (calc, n) => calc.expensiveCalculation(n))
    .subscribe(console.log, undefined, () => console.log('3rd done'));
}, 5000);